Posted by Brett Sheppard on July 25, 2010
Making Hadoop Accessible for Enterprise Developers and Analysts
Karmasphere offers front-end client software that enables developers and analysts who are not necessarily Hadoop specialists to develop, debug and deploy Hadoop jobs to virtually any private, public or hybrid Hadoop cluster. You can download the NetBeans or Eclipse versions of the no-cost Karmasphere Studio Community Edition, or apply for a limited beta of the just-announced Karmasphere Studio Professional Edition and Analyst Edition, at Karmasphere’s software download page. Editor’s note: This is a reprint of an article that first appeared on Big Data News.
Karmasphere focuses on making Hadoop accessible and usable for developers and analysts who are looking to deliver business value from Hadoop for advanced, highly scalable analytics but who are not necessarily Hadoop specialists. Karmasphere was founded in 2005, and received US$5 million in venture capital funding in March 2010 from two Tier 1 venture capital firms, Hummer Winblad Venture Partners and U.S. Venture Partners. Karmasphere has not yet released pricing or named customers.
I had the opportunity to talk with Karmasphere founders Martin Hall and Shevek Mankin, in a pre-release briefing prior to the 2010 Hadoop Summit in Santa Clara, CA. Previously, I first met Martin as a panel moderator at the SDForum Analytics Revolution conference, where Karmasphere exhibited and presented.
Karmasphere helps insulate enterprise line-of-business and corporate technical and business users from Hadoop complexity. Karmasphere has coined the term “Big Data Intelligence” to focus attention on making Hadoop usable for wider enterprise adoption.
Karmasphere Software Editions
In their product announcement, Karmasphere previewed their Karmasphere Studio Professional Edition, Karmasphere Studio Analyst Edition and Karmasphere Client, all of which are based on Karmasphere’s underlying Application Framework.
About the new editions and client software:
- Karmasphere Studio Analyst Edition (generally available in Q3 2010) supports Hive and delivers a SQL-like interface for ad hoc analysis of Big Data.
- The Professional Edition is designed for the commercial development, debugging, deployment and analysis of MapReduce applications on Hadoop clusters, with additional functionality not available in the Community Edition to mask the complexity and enable greater efficiency for MapReduce jobs, including some features for query optimization. One of its nice features is the ability to use point-and-click to debug job performance on the cluster.
- The Karmasphere Client works on Windows, Mac and Linux systems, and supports Hadoop releases 18, 19 and 20, unlike the standard Hadoop client that is version-specific and does not currently support Windows (perhaps not surprisingly, given that the initial inspiration for Hadoop came from Google’s MapReduce and File System research, and that Yahoo! has been a leading contributor to the Apache Hadoop project).
While provisioning cloud clusters is not Karmasphere’s primary corporate vision, their software does include nice tools to switch data volumes and analytics jobs among different public and private clouds. For example, a company can test analytic models in a public cloud using sample data, and then convert back to their own data centers to run those models using confidential customer data.
The press release includes two partner quotes, from Peter Sirota, General Manager of Amazon Elastic MapReduce at Amazon Web Services, and Dr. Amr Awadallah, co-founder and chief technology officer of Cloudera. Karmasphere is integrating its product offerings with solutions and services from Amazon Web Services and Cloudera.
Through some clever software engineering and a fair amount of personnel investment, Karmasphere has enabled their software to bypass the “every node can talk to every note” restraints of the Hadoop architecture to work within hub-and-spoke Secure SSH firewalls, by rewriting the network layer for the endpoints, and thereby taking the “Hell” out of Secure SHell (SSH) for Hadoop users.
Fit in the Cloud Ecosystem
Companies like Karmasphere form an important component of the Hadoop ecosystem, to enable technical and business users who are not active participants in the Apache Hadoop project to deploy Hadoop seamlessly into corporate and line-of-business enterprise environments.
By focusing on Hadoop, Karmasphere is distinct from more horizontal cloud provisioning platforms, such as CloudSwitch, the winner of the GigaOM Structure LaunchPad awards, or Zettar, another startup that presented in that LaunchPad. Another difference is that those platforms are largely focused on streamlining cloud provisioning and enabling inter-operability across different types of clouds (an important problem in-and-of itself), which largely represent enablement of server-to-server data transfers. Meanwhile, Karmasphere focuses on the Hadoop cluster to human interfaces.
Likewise, Karmasphere is distinct from another GigaOM Structure LaunchPad presenter and Hadoop Summit exhibitor, Datameer, which focuses on an Excel-like interface for business analysts.
And, in case you missed the earlier reference to Cloudera in today’s Karmasphere press release, the Karmasphere user interface sits in front of Cloudera, Amazon Web Services or other cloud providers, and does not compete with those companies.
Has your organization tested Karmasphere’s software, or used Karmasphere in a production environment? If so, what have you found?