Focus on Big Data

I’ve been a long time data geek and thus have been spending a lot of time looking at the startup opportunities in the field of Big Data. Big Data as a topic covers instances where either an application or a database stores an abnormally large amount of data; think zetabytes and petabytes. The discipline is broad and includes technologies and approaches that focus on storage, caching, data distribution and partitioning, analytics and predicting the future. 

The social web has been fertile ground upon which many Big Data startups have been planted. Open source movements like Hadoop, GridGain and Pentaho have iterated rapidly and have been used primarily to wade through the terabyte of data generated daily by social computing platforms.

Big Data technologies are already in use within the government, healthcare, financial services and retail sectors with more to follow.

That’s why today we begin coverage on the Big Data ecosystem by aggregating great content from thought leaders and practitioner from around the world. You can read more in the Big Data section of this site and stay tuned for more on Big Data startups.

Don't Kill a Good Idea

The Blog Comment is Dead, Long Live the Blog Comment

We are almost entering the year 6 AT (after Twitter) and I think it’s time to reflect on the impact that this little tool has had on the Blogosphere.

First, microblogging has reduced the number of hobbyist bloggers. Those who once blogged have been whipped a state of retweet frenzy. Why take the time and make the effort to write a thoughtful post when I can retweet someone else’s thoughtful post; rinse then repeat?  

The 140 character limit has forced us to economize our thoughts but at the expense of clarity turning many ex bloggers into headline specialists, abbreviated linguists and link shortening specialists. After all what is a twitter post other than Blurb <link> Blurb <hashtag>. 

The channel is noisy - perhaps too noisy - with signal reception problems than can only be solved with client side technologies that filter out spam, porn and other unsolicited nonsense. A philosopher majoring in Yogi Barraisms might paraphrase with: “Nobody goes there anymore ‘cuz it always too crowded.” - Or just too noisy.

Ironically, this post will be advertised through Twitter, hopefully triggering numerous retweets and a cacophony of static in the twitterverse. Is this a great universe or what? 

The wonderfully crafted, cynical, sarcastic but truthful blog comment - once a work of art - is now Twitterverse casualty. With plug-ins Blogs can link to a Facebook like or comment, solicit comments via tweet or a one-click G+. Economical but at the expense of noise inducing hyper-shares.

While the Blog is alive and well, the blog comment has died a slow death - and we’ll miss it when it’s gone - So Long Live the Blog Comment…you were here for such a short while and we hardly knew ye. 

Social Media Brand is the New Black

There is an old saying probably invented by a PR hack in 1950s New York. It goes: “If you don’t tell your story then somebody else will.” I ran across it  in a post by Justine Musk; if you haven’t read her Blog then head on over she’s brilliant, but that’s another story and I digress. 

The quote is as true today as it was when it was first minted and the notion that “it’s your story go tell it” struck a chord in the social media music box that lately has become my mind. You see, there are too many opportunities within the socialverse today for your customer to tell your story for you - and get it all wrong. 

Products like Angie’s List, Yelp, Open Table, Four Square allow the customer to start a conversation about you. Positive word-of-mouth advertising is wonderful; a mediocre rating or unflattering commentary can be damning.

How do you control the damage once you’ve been tagged mediocre?  What do you do if the social conversation relegates you to a niche you never intended to occupy? It’s impossible to control the social conversation. So what’s the strategy?

Be the first to frame the conversation.

Extend your web presence through social media. Stake a claim to your corporate name on Facebook and LinkedIn. Microblog with Twitter and/or Tumblr . Develop an entrepreneurial persona by blogging passionately about topics related to your areas of expertise. Carve and occupy the niche you choose, build your social brand. 

The only way to keep others from telling your story incorrectly is to be there months ahead of them with the twenty-four carat gold original. 

Tweet-On.

Education: The Disruptive Power of Zero

The secret of success in any startup is to provide a product or service needed by thousands and, via technology, deliver it at the lowest possible price point. The product doesn’t have to be functionally equivalent or as powerful as its competitive counterparts, but in order to displace them it must cost less. Additionally, lowering the price point also opens the product to a larger market often left untapped by the higher priced vendors.

Most of us would agree that higher education is overdue for a make over and most students would flock to an institution that offered a quality education at an affordable price. Last week Apple Computer tilted the playing field through the introduction of iBook Author and its integration with iTunes University. 

iBook Author lets anyone create rich, multimedia course content for free. Textbook authors can then publish their content to iTunes University where students can search, download and consume university quality courseware for much less than the price of a college credit. Many courses from prestigious universities like MIT, Cal Berkley and Stanford are free.

Undergraduate college students, on average, pay approximately $1,200 per year for textbooks; graduate students pay even more. With iBook Author, educators can take control of text book creation, eliminating the large publishing company middleman and reduce the price of textbooks. 

Companies like Udacity and Udemy have constructed service offerings that enables community created content and university created content to be published online and consumed en mass. The on-to-many architecture of the Internet allows thousands of students to take the same course at the same time with the same set of instructors and moderators. These economies of scale can’t be duplicated by traditional brick and mortar universities.

One piece of the puzzle is missing though: accreditation. While online institutions like the University of Phoenix offer college credit at higher prices than the disruptive startups they also confer degrees and certifications; which for most is the driving force behind taking a college or continuing education course. 

Only when high quality, low cost courseware is integrated within cirricula offered by accredited degree conferring institutions will we see the true disruptive power of zero emerge. 

Somewhere a startup is brewing their own version of Higher Education 2.0. and in the near future a college education will be made available to a much larger economic demographic; and that my friends will be a very significant turn of events. 

Startup Mantra: GSD

Woody Allen once said that 80% of success is just showing up. Nike tells us to “Just Do It”. I prefer to: make plan, prioritize plan, execute plan…or just simply: Get Shit Done.

GSD

Software Technology, Job Disruption and Systemic Unemployment

How many programmers does it take to eliminate 10,000 jobs?  It’s probably a lot fewer than you think. Over the past decade entire sectors of the world economy have been permanently altered by the disruptive power of software driven startups. 

The most notable example is probably Craigslist a small San Francisco startup. Founded in 1995 by Craig Newmark, Criagslist grew from a small apartment based dot-com into the industries leading purveyor of classified ads and in the process toppled the newspaper industry’s control over the classified market.  The loss of the lucrative classified revenue stream coupled with online news outlets caused approximately 450 newspapers to shutdown while shedding over 40,000 jobs since 2007. Craigslist employs approximately thirty with a technical staff of twelve. 

In late 2011 Borders finally closed its doors laying off approximately 16,700 jobs. The cause: Amazon.com and the evolution of the eBook embraced by Apple, Amazon and Barnes and Noble. Add to this the fact that over 1,500 independent bookstores have closed in the last seven years. 

A slumping economy coupled with software delivered online ads has forced ad agencies to cut over 125,00 jobs since 2008 while Internet ad revenues rose to nearly $15 billion in first-half of 2011. 

In their book Race Against the Machine authors Erik Brynjolfsson and Andrew P. McAfee write that during the last recession the economic downturn prompted many businesses to look harder at substituting technology for people, where possible. Since the end of the recession in June 2009, they note, corporate spending on equipment and software has increased by 26 percent, while payrolls have been flat. 

In an August, 2011 puff piece Marc Andressen explains why software is ‘eating the world’. He describes a dozen industries disrupted by ‘software company’ startups indicating that more productive, free software tools coupled with the power and scale of cloud computing is responsible for this trend. And the trend will accelerate and  so will the rate of ‘forced’ unemployment.

Perhaps the software startups should be thought of as serial job killers capable of putting hundreds of thousands out of work within a matter of years.  Classical economists would have looked at this as a healthy state of events eliminating labor inefficiency in one sector of the economy freeing and then deploying that labor (ceteris paribus) to other sectors where it could be put to more productive use. But, that’s not been the case.

A self proclaimed serial job killer John Jazwiec notes that “any job that can be eliminated though technology or cheaper labor is by definition not coming back. The worker can come back. They most often come back by being underemployed. Others upgrade their skills and return to previous levels of compensation. But as a whole, the productivity gains over the last twenty years, have changed the landscape of what is a sustainable job.”

So while inexpensive labor may, in some cases, be a substitute for productivity enhancing technologies it is clear that software’s voracious appetite and the automation of all tasks that were once performed by humans is the foundation of systemic unemployment

The cure? Well that will only come in another, better educated America.  ”Where does the before-gainfully employed professional learn how to compete for jobs?”, writes Jazwiec. “Monster dot com? LinkedIn? The unemployment office? What infrastructure exists to retrain professionals to compete in a free agent job market, where only creative unique value contributors thrive?”

Indeed. 

In the meantime there is hope. If you’d like to land a new job or possibly future proof your employment prospects just get online and learn to program. Sites like Code Academy, Udema and Udacity may make training and retraining the underemployed more productive and more affordable than every before.  

Sources:

1. Newspapers Layoffs  

2. Ad Industry Job Losses 

3. Craigslist Fact Sheet

4. Software eats the world

5. Serial Job Killers: 

New Mechanisms in a Brand Driven Economy - Part 1

It’s getting pretty hard for brand managers to ignore the effect social media has had on their marketing and advertising budgets. Internet media buys via paid search, private placement of ads or through the corporate activity stream and social media landing pages have taken center stage in the B2C firm’s marketing department. Over the last 18 months we’ve used many of these same outlets to communicate with our customer base; we’re an IT firm selling B2B and delivering SharePoint, CRM and Business Intelligence services to our Mid-Atlantic customer base. 

Another trend we’ve witnessed in our customer base is the deemphasis of SEO and search word buys in favor of social media and social search advertisement placement. On Facebook, for instance, you can dial in the demographic a little more granularly targeting ad buys to fans, would-be fans an the general public, tuning and measuring the message as you go. 

The potential for a positive feedback loop is enormous. Messages delivered to the customer base can be monitored for activity, commentary, likes and shares. Social CRM tools have emerged that are tuned into social networks and provide a multichannel monitoring device which enables the savvy marketing firm to identify key customers and product evangelists. 

Not suprisingly, our clients are iniating fewer direct email campaigns, instead favoring  messages placed into the Facebook, Twitter and LinkedIn social streams. Permission based marketing has morphed from push to pull with an emphasis moving towards the acquisition of social identities and not email identities. 

Unfortunately the tools used to manage this new form of brand and identity management often focus on one or two social media techniqes and have few abilities to integrate. There are however a number of best of breed products and techniques that focus on social media marketing’s key mechanisms: the Activity Stream, Ad Placement, Events & Calandering, CRM and Analytics. 

We’ll cover these in subsequent posts so stay tuned, and as always, your feedback is appreciated.

Thought Experiment: Ending Email as We Know It.

The number of deplorable email products on the market today is staggering. Whether you use a private email service at your firm like Microsoft Exchange or you’re using a cloud based product like GMail or Yahoo Mail you’re faced with the same headaches: SPAM, volumes of interesting but unsolicited email, organizational headaches, attachments, folders, contact management and, inevitably, a 1990s user interface. 

But we’re at a technological inflection point where the convergence of social media, cloud storage, ad hoc peer-to-peer technology and mobile, always on platforms should provide us with a means to reinvent the email.

I suspect that among certain demographic groups the amount of email among trusted friends has dropped dramatically due primarily to products and applications that connect to Facebook, Twitter and SMS. But that gives rise to another organization problem: where was that last communication from my friends: in a Tweet, a LinkedIn email, and SMS on my phone or in an email.

Then there is the attachment problem; physically linking a file and not a reference to a message. How do we ween our peers from the habit of attaching multi-megabyte files to emails instead of referencing them through a shared public or private location accessible though a URL.

The sheer number of ways in which we can communicate has led to social media unification products like TweetDeck which tries to combine all social media streams into a singular interface. It’s handy for real time streams but it lacks access to a historical memory and provides rudimentary, agonizingly slow search. 

Facebook has created Timeline that does a great job of cataloging communications across the time dimension but provides little in the way of categorization or search. That works well if you only use Facebook to communicate to friends and family. 

Google is attempting to solve this problem by integrating G+ with Gmail and Google Search. Yet the G+ APIs are immature and provide no way of integrating multiple social streams, unless of course they all terminate as a plug-in to G+. 

On the enterprise side of the equation we have the eponymous Jive and Yammer as well as Chatter from Salesforce.com. Each attempts to manage micro blogging, the creation of trusted groups, security, cloud based attachments and a set of rich APIs that support the integration with intranet, extranet and internet properties.

Unfortunately these enterprise tools are islands. Each survives through differentiation, price disruption and through an integrated product offering. They don’t integrate with each other though and this poses a problem that email just doesn’t have.

But our email infrastructure does possess: 

• a well documented and standardized protocol (iMAP and SMTP) that all email services and service consumers understand.
• an addressable endpoint - an email address that supports delivery from any sender.
• the ability to deliver message payloads across heterogeneous service providers - due to the protocol and the adherence of the service community to that protocol
• the ability to forward, attach and reference content within the message payload.
• an infrastructure supportive of various clients capable of consuming messages.
• the capability to thread discussions
• a secure store requiring authentication to allow access

So how will social media kill email. It won’t be easy, but in future posts I’ll try to outline what I believe can be done to support the aggregation of the social stream.

White Boards and Power Point Decks

I work with teams of varying size and we work together to build solutions on many different technology platforms. I often come across product or project managers who steer every design or strategy meeting with a PowerPoint deck. I’m not suggesting that these sessions should be agendaless; they are necessary evils intended to keep participants focused and on track - but maybe that’s not in the best interest of the design team.

Great design sessions tend to be random, cover dozens of topics and are an important forum for the team to interact, weigh in, evaluate options and prioritize goals. PowerPoint decks are a linear crutch. Design by PowerPoint negatively affects the artifacts generated by these sessions. 

Instead of ‘death by PowerPoint’ give me 30 square feet of whiteboard, a dozen dry erase markers and a camera. White boards are visual, they support developing a collage of design artifacts. Several team members can use the white board simultaneously, it’s easy to retrace (or erase) rapidly. A digital camera quickly captures the whiteboard contents and acts as the design team’s memory.

Lately we’ve integrated iPhones and iPads into our design sessions. With one quick snap of a camera, we can rapidly share design diagrams among the team members…and start fresh with a clean board.

Improve your designs, ditch PowerPoint.

Currently Reading: Universal Principles of Design...

This is an excellent read for designers and innovators. This is the 2nd edition which includes several newly added concepts that spread out across a 272-page book. This is a great reference book for anyone building tech products; especially for designers, engineers, architects, and other creative professionals.

Infographic Saturday: The Life of Steve Jobs

I believe in the power of visualization and that is certainly embodied in the incredibly creative infographics that have been published on the web in 2011. The thumbnail on the right depicts the Best Infographic of 2011, by i guess, the Infographic Society of America or some other august body. Click through the graphic for the full sized version. Enjoy.

CTO or Engineering VP - What's the Difference?

I've worked in quite a few startups where during the early days of the company our small teams shared quite a few responsibilities. During crunch time everyone pitched in to either build, test, demo or sell the product. After gaining traction, some funding and a few customers the teams invariably grew and with growth came the realization that team roles and job responsibilities needed to be more clearly defined. It was never an easy task.

One of the problems we encountered over and over again was defining exactly what a Chief Technology Officer, or CTO, was and how that differed from a Vice President of Engineering. At the highest level the CTO is chartered with defining the technology vision of the company. The VP of Engineering is tasked with building and managing the team that can execution the CTO's vision.

At his or her core, the VP of Engineering is a manager. They are responsible for devising the engineering team structure, have the ability to recruit, know how to build a team that scales and imparts on that team the necessary discipline to get the job done. The ideal VP of engineering is an engineer with a firm grasp of software engineering methods and project management techniques. It is this individuals responsibility to make sure that the job is also done right.

The CTO is a strong technologist capable of looking at the product or service at a higher level of abstraction. The CTO is an architect while the VP of Engineering is the crew foreman. The CTO is a renaissance person, probably more of a high level thinker when compared to the engineering team. The CTO not only owns the vision but is capable of communicating that vision to partners, investors and customers. The CTO is often the product or service spokeperson.

Building technology products is a team sport and more often than not the VP of Engineering and CTO can be thought of as co-captains. They must work together and complement each other.  Both should report directly to the COO, although I've seen org charts that had the VP of Engineering reporting to the CTO.

More often than not the CTO will be a founder or cofounder of the company. Initially, technology startups are driven by the technologists and ultimately it's one of the cofounder's roles to step up and assume CTO responsibilities. One of the first jobs that the COO and CTO perform together is choosing or recruiting the VP of Engineering.

Building your Team

Almost every startup begins with only a few founders; more often than not a company is started by a single individual who has the vision and technical know-how to bring a product to market. Here's some thoughts on how a lone gunman can build his startup team:

• Your first several hires are important. What are they?  Who are they? Plan then execute.

• Find complements. If you’re a technical genius then partner with a sales or marketing guru.  If your background is sales and you have an amazing technical idea find a technical genius to help implement your vision. Think Wozniak and Jobs or Brin/Page and Schmidt.

• When picking a leadership team keep in mind that this group will run day-to-day operations, track financials, market/sell and develop the products and services you’ll be delivering.

• When recruiting your team be an evangelist and a cheerleader. Know when to get in the pulpit and when to put on a skirt.

• When forming that core team provide each member with performance incentives in the form of equity and/or cash. While shared equity isn’t necessarily shared vision; it goes a long way to ensure that the start-up team is focused on the same economic targets.

Every Pitch Should Tell a Story

I’m often asked to listen to pitches describing what somebody believes their company, product or service is all about. Mostly these are excruciatingly dull and painful episodes on par with a trip to the dentist. On the rare occasion that I see or hear something extraordinary it is usually because the speaker has decided to tell an engaging story. The great presentations engage with story lines that transcend pitch. Try to stop watching the presentation below…

Chris Anderson: How web video powers global innovation on Prezi