Next hottest tech career: Data Scientist

I attended two interesting technology conferences last fortnight, and listened to two energetic speakers. At the SAP Forum Mumbai, we had an invigorated Steve Lucas, Executive Vice President and General Manager, SAP Database & Technology telling the audience why SAP was a better choice for enterprise databases. And at Teradata Universe 2012,  the audience listened attentively to Stephen Brobst, Chief Technology Officer, Teradata Corporation, who gave a highly lucid explanation of Big Data.  Both spoke about the role of the Data Scientist, and why every organization should have one on its employee roster.

So what does a data scientist do, what academic qualifications do you need to qualify as one, and why is the data scientist going to be much sought after tomorrow? For that matter, is this just a replacement term for ‘data analyst’?

Going by one definition, a data scientist is a person who excels at analyzing large amounts of data, to help a business gain competitive edge.

A example is the call detail record or CDR generated by a telecom company. A data analyst would scan the CDR and then advise the business what services to introduce based on usage trends and the movement of users in the network.

The data scientist is obviously associated with Big Data and analytics. Everyone knows that big data or high volume data was always around; the difficult part is identifying the most useful parts of this data, and establishing patterns and trends, so that one could make smart business decisions.

That’s what a data scientist will do.

“A data scientist need not be a programmer, but he should be able to extract value from the data,” said Teradata’s Stephen Brobst. “But he should be able to use tools to discover new insights from this data.”

A data scientist would have a combination of analytical and statistical skills, and would be adept at machine learning and data mining. And that gifted person would also have experience with algorithms and coding.  But the data scientist would also need to have a special ability of explaining business trends to others in the organization.

SAP’s Steve Lucas said a data scientists must have a degree in economics, statistics, or mathematics (or all three preferably).

“They should also have spent a good part of their career with practical applications like analytics, predictive technologies, data management, data modeling, statistics, forecasting etc. I also see that Wall Street seems to be a good source for data scientists to hire from,” said  Lucas.

Lucas opines that companies are becoming data hoarders — not knowing what to do with so much data. “It’s a lot of noise and not much signal,” he says. “We need to look for new signals to understand our business.”

Well, it looks like we’ll soon have a new position in corporations, designated chief data officer (CDO) or equivalent. But rather than having a single person responsible for this task, it should really be a team of people. For instance, many organizations have in-house or external social media teams to manage their social media pages; these teams scan posts and send a customer sentiment report back to marketing (customer sentiment analysis).

This team could evolve into a data analysis team, and be led by data scientists. The output from this team would be intelligence reports that advise the business on what directions it needs to take, based on market and customer dynamics.

Well, financial analysts and consultants are doing this on a larger scale today. But I’m suggesting an in-house team, working full time. The task is also becoming easier with the availability of data visualization tools.

Can HP ‘Invent’ again?

I was sad to learn that HP, once a technology stalwart, is slashing 27,000 jobs by 2014 (and 9,000 this year). That’s a clear indication of a sinking ship. No doubt, this move will slow down its fall. HP CEO Meg Whitman recently told analysts that HP is going to invest more in research, development and innovation. You may recall a past HP tagline that was simply, ‘Invent’. And HP was once spoken in the same breath as companies such as Xerox. HP missed out on many opportunities. Its successive leaders struggled to turn around the company. True, it excels in areas like printing, networking and servers, but these are the few aces left in HP’s hand. Along the years, HP acquired illustrious companies like Compaq, 3Com, Palm, and Tandem, but took a long time to integrate those companies’ product lines and to produce top selling solutions based on that IP. There were ‘bad’ acquisitions like Palm, and delayed decisions about producing tablets (HP Slate).  And there were ‘good’ acquisitions like 3Com, Compaq and 3PAR that gave HP a stronger position in networking, compute, and storage. But now that HP is getting back to innovation & research, here are some areas it should focus on.

Firstly, here’s what HP should not try to be. HP should not try to be like Apple, a producer of well designed consumer products. HP should not try to be like Samsung, LG, or Sony, who lead in consumer electronics. And HP should not try to be like Google, Facebook or any of the Web 2.0 companies. Because it is not an internet company.

Traditionally, HP has been a research company that built innovative, hi-tech and useful products like pocket calculators and laser printers. It could have continued in this line and built industry-specific solutions such as label printers, card readers and bar code scanners. The Imaging and Printing Group at HP missed the digital photography bus. Yes, HP did acquire an online photo printing company, and tried to offer customized printing solutions (I can’t imagine my photo on a coffee mug or t-shirt).  And HP made a recent effort with cloud printing services.

Why didn’t HP get into medical systems like GE, Siemens and Philips? With its clout in imaging, it could have produced diagnostic imaging systems (X-ray machines, CAT/MRI scanners) for healthcare.

Now here’s what HP can and should do.

It’s not too late for HP to pursue today’s hottest technologies in the business world: cloud services, big data, BI/analytics. Look at what EMC is doing. EMC was just a storage company that later created the networked storage category. Today it manufactures big data analytics appliances (Green Plum). Oracle, once a maker of only RDBMS products is doing likewise (after its acquisition of Sun). And then there’s Dell, once a PC company that reinvented its self and is now selling end-to-end enterprise solutions and also services.

HP has focused too much on hardware and I think it needs to tone its software muscles. Dell is quietly setting up a software division and I think HP should do likewise. Software is the key to smarter network management tools, BI/Analytics and big data. And HP can embrace open frameworks like Hadoop (just as EMC did).

Another possible area is hardware-based security (embedded in silicon), video surveillance, and biometrics (HP Imaging).

HP could boost its consulting and global services business, an area long dominated by IBM.

HP needs to have a vertical focus and make products for industries like banking & insurance, healthcare, retail & logics, manufacturing, travel and government.

Clearly, the workforce at HP needs to tilt towards R&D. HP should think about setting up a separate research company on the lines of Xerox Parc.

There’s hope yet for HP. It needs to ‘re-invent’ itself soon.