从草根工程到行业标准 一个开源小项目的进化神话：j9九游真人游戏第一平台

There are countless open source projects with crazy names in the software world today, but the vast majority of them never make it onto enterprises’ collective radar. Hadoop is an exception of pachydermic proportions.如今的软件界具有数不清的开源项目，它们享有可怕的名字，但其中的大多数根本都没进过企业的法眼，只有Hadoop是个值得注意。Named after a child’s toy elephant, Hadoop is now powering big data applications at companies such as Yahoo YHOO 2.57% and Facebook FB -0.46% ; more than half of the Fortune 50 use it, providers say.Hadoop的名字源于一个小孩的玩具，如今已被用作雅虎（Yahoo）和Facebook等公司的大数据程序中。供应商回应，《财富》50强劲中有半数以上的公司都在用它。

The software’s “refreshingly unique approach to data management is transforming how companies store, process, analyze and share big data,” according toForrester analyst Mike Gualtieri. “Forrester believes that Hadoop will become must-have infrastructure for large enterprises.”根据弗雷斯特研究公司（Forrester）分析师麦克o瓜尔蒂耶里的众说纷纭，这个软件“在数据管理上使用了令人耳目一新的独有方法，转变了各公司存储、处置、分析和共享大数据的方式。”弗雷斯特指出Hadoop不会沦为大型企业不可或缺的架构。

Hadoop在2012年的全球市值为15亿美元，而到2020年，人们估算它的价值将不会超过502亿美元。Globally, the Hadoop market was valued at $1.5 billion in 2012; by 2020, it is expected to reach $50.2 billion.一个草根的开源项目最后出了行业标准，并不是一件常有的事。

Hadoop是如何做的？It’s not often a grassroots open source project becomes a de facto standard in industry. So how did it happen?“一个享有急迫市场需求的市场”‘A market that was in desperate need’分析公司RedMonk联合创始人和首席分析师史蒂芬o奥格雷迪说道：“Hadoop是由基础的差异化技术、取得许可的进源代码库和迫切需要解决问题数据发生爆炸的方法的市场三者融合构成的凑巧。从这一点上来说，它的顺利并不令人车祸。”“Hadoop was a happy coincidence of a fundamentally differentiated technology, a permissively licensed open source codebase and a market that was in desperate need of a solution for exploding volumes of data,” said RedMonk cofounder and principal analyst Stephen O’Grady. “Its success in that respect is no surprise.”这个软件的创造者是道格o卡廷和麦克o卡法雷拉。

它与许多其他发明者一样，都是应需而生。2002年，两人都在为一个叫作Nutch的开源搜索引擎工作。卡廷说道：“我们获得了一些进展，在小范围的机器上运营了它。

但我们依然不确切要怎么不断扩大它的用于范围，让它像谷歌（Google）一样被成千上万的机器用于。”Created by Doug Cutting and Mike Cafarella, the software—like so many other inventions—was born of necessity. In 2002, the pair were working on an open source search engine called Nutch. “We were making progress and running it on a small cluster, but it was hard to imagine how we’d scale it up to running on thousands of machines the way we suspected Google was,” Cutting said.之后旋即，谷歌就谷歌文件系统（Google File System）和MapReduce公开发表了一系列学术论文，卡法雷拉说道：“于是我们迅速就确切了，Nutch必须享有一些类似于的架构。”Shortly thereafter Google GOOG -0.34% published a series of academic papers on its own Google File System and MapReduce infrastructure systems, and “it was immediately clear that we needed some similar infrastructure for Nutch,” Cafarella said.卡廷说明道：“谷歌处置问题的方法与众不同，十分简单。

”目前为止，人们一般来说指出“你必须为每一个想已完成的分布式任务创建专门的系统”，而在这一点上，谷歌获取了一个标准化的自动化架构来已完成分布式计算。卡廷说道：“它需要处置分布式计算中的那些艰难的部分，如此一来，人们就可以专心撰写自己的程序。”“The way Google was approaching things was different and powerful,” Cutting explained. Whereas so far at that point “you had to build a special-purpose system for each distributed thing you wanted to do,” Google’s approach offered instead a general-purpose automated framework for distributed computing. “It took care of the hard part of distributed computing so you could focus just on your application,” Cutting said.卡廷和卡法雷拉【如今分别是Cloudera首席架构师和密歇根大学（University of Michigan）计算机科学和工程专业的助理教授】告诉，他们得作出自己的架构——不仅是为了Nutch，也是为了教化其他业内人士——他们明白自己想要把它制成开源。Both Cutting and Cafarella (who are now chief architect at Cloudera and University of Michigan assistant professor of computer science and engineering, respectively) knew they wanted to make a version of their own—not just for Nutch, but for the benefit of others as well—and they knew they wanted to make it open source.卡廷说道：“我不讨厌商业的那些事，我只是个做技术的。

我讨厌写出代码，与同事合作解决问题，完备我们的产品，而不是试着把它变卖。我更加不愿告诉他别人‘这一点上它做到得不俗，那一点上过于差劲了，或许我们可以改良一下。’需要当一个完全真诚的人感觉很好，而在商业环境中，你很难维持这一点。

”“I don’t enjoy the business aspects,” Cutting said. “I’m a technical guy. I enjoy working on the code, tackling the problems with peers and trying to improve it, not trying to sell it. I’d much rather tell people, ‘It’s kind of OK at this; it’s terrible at that; maybe we can make it better.’ To be able to be brutally honest is really nice—it’s much harder to be that way in a commercial setting.”但是这两人告诉，这项技术一旦取得成功，将不会具备极大的潜力。卡廷说道：“如果我没有辨别拢，这是项很简单的技术，许多人都想要，那我就能付我的房租了，我们的初创公司也就没有那么大风险了。”But the pair knew that the potential upside of success could be staggering. “If I was right and it was useful technology that lots of people wanted to use, I’d be able to pay my rent—and without having to risk my shirt on a startup,” Cutting said.对卡法雷拉而言，“将Nutch开源，部分原因是想看见搜索引擎技术挣脱少数几家公司的独占，但这也是一项战略要求。

如此一来，我们就最有可能获得来自大公司的工程师的协助。我们特地自由选择了一个能让其他公司最精彩地参予进去的开源许可。”It was a good decision. “Hadoop would not have become a big success without large investments from Yahoo and other firms,” Cafarella said.这是一项英明的要求。卡法雷拉说道：“如果没雅虎和其他公司的大量投资，Hadoop有可能会这么顺利。

”‘How would you compete with open source?’“没有谁拼得过开源产品？”So Hadoop borrowed an idea from Google, made the concept open source, and both encouraged and got investment from powerhouses like Yahoo. But that wasn’t all that drove its success. Luck—in the form of sheer, unanticipated market demand—also played a key role.所以Hadoop借出了一个来自谷歌的点子，把这个概念开源，然后获得了雅虎等大公司的希望和投资。但这并不是造成它顺利的全部因素。运气——几乎没预想到的市场需求——也在其中起着了关键因素。“I knew other people would probably have similar problems, but I had no idea just how many other people,” Cutting said. “I thought it would be mostly people building text search engines. I didn’t see it being used by folks in insurance, banking, oil discovery—all these places where it’s being used today.”卡廷说道：“我告诉其他人可能会遇到类似于的问题，但我不告诉竟然这么多人都有。

我实在大部分用户都会是文本搜索引擎的开发人员，可没有预料到许多专门从事保险业、银行业和石油勘探业的人也不会用它——它早已在这些领域获得了应用于。”Looking back, “my conjecture is that we were early enough, and that the combination of being first movers and being open source and being a substantial effort kept there from being a lot of competitors early on,” he said. “Mike and I got so far, but it took tens of engineers from Yahoo several more years to make it stable.”叹往昔，卡廷说道：“我猜中我们积极开展得充足早于，作为第一批推动者，我们做到的又是开源产品，也代价了大量希望，这一切让我们与许多早期竞争者区分了出去。麦克和我早已研发了很久，不过来自雅虎的几十位工程师又花上了好几年时间才让这个架构显得平稳。”And even if a competitor did manage to catch up, “how would you compete with something open source?” Cutting said. “Competing against open source is a tough game—everybody else is collaborating on it; the cost is zero. It’s easier to join than to fight.”卡廷回应，即便有竞争者想迎头赶上，“你又怎么能拼得过开源产品呢？和开源产品竞争是十分艰难的事——其他所有人都会为它做到贡献，他们没成本。

重新加入他们比对付他们更容易。”IBM IBM -0.24% , Microsoft MSFT -1.30% , and Oracle ORCL 0.00% are among the large companies that chose to collaborate with Hadoop.国际商业机器公司（IBM）、微软公司（Microsoft）和甲骨文（Oracle）就在那些自由选择同Hadoop合作的大公司之佩。Though Cafarella isn’t surprised that Web companies use Hadoop, he is astonished at “how many people now have data management problems that 12 years ago were exceedingly rare,” he said. “Everyone now has the problems that used to belong to just Yahoo and Google.”尽管卡法雷拉并不怪异网络公司不会用于Hadoop，但他回应，他对“这么多人都遇到了12年前十分少见的数据管理问题”深感愤慨。

“曾多次只有雅虎和谷歌才不存在的问题，现在后遗症着每一个人。”Hadoop represents “somewhat of a turning point in the primary drivers of open source software technology,” said Jay Lyman, a senior analyst for enterprise software with 451 Research. Before, open source software such as the Linux operating system were best known for offering a cost-effective alternative to proprietary software like Microsoft’s Windows. “Cost savings and efficiency drove much of the enterprise use,” Lyman said.信息技术研究公司451 Research的企业软件高级研究员杰伊o莱曼回应，Hadoop代表了“一种开源软件技术的主要推动者的转折点。

”在这之前，开源软件比如Linux操作系统，是因为获取了微软公司Windows这类专有软件之外的合算自由选择，才声名鹊起。“企业用于它们，大部分都是出于节约成本、提高效益的考量。

”With the advent of NoSQL databases and Hadoop, however, “we saw innovation among the primary drivers of adoption and use,” Lyman said. “When it comes to NoSQL or Hadoop technology, there is not really a proprietary alternative.”不过，随着非关系型数据库（NoSQL）和Hadoop的经常出现，莱曼说道，“我们看见使用者中经常出现了有创意之举的推动者。非关系型数据库和Hadoop技术并不确实归属于专有技术之外的其他自由选择。

”Hadoop’s success has come as a pleasant surprise to its creators. “I didn’t expect an open source project would ever take over an industry like this,” Cutting said. “I’m overjoyed.”Hadoop的顺利对创造者来说是一种惊艳。卡廷说道：“我没想起一个开源项目需要像这样引导着行业。

我太高兴了。”And it’s still on a roll. “Hadoop is now much bigger than the original components,” Cafarella said. “It’s an entire stack of tools, and the stack keeps growing. Individual components might have some competition—mainly MapReduce—but I don’t see any strong alternative to the overall Hadoop ecosystem.”它依然发展得如火如荼。

卡法雷拉说道：“相比最先的组件，Hadoop现在可观多了。它早已出了一整套工具，而且还在之后扩展。单个的组件或许不会遭遇竞争者——主要是MapReduce——但我没见过需要代替整个Hadoop系统的强劲输掉。

”The project’s adaptability “argues for its continued success,” RedMonk’s O’Grady said. “Hadoop today is a very different, and more versatile, project than it was even a year or two ago.”RedMonk的奥格雷迪说道，这个项目的适应性“需要让它大大顺利。现在的Hadoop十分与众不同，相比一年或者两年前，它的功能更为强劲了。”But there’s plenty of work to be done. Looking ahead, Cutting—with the support of Cloudera—has begun to focus on the policy needed to accommodate big data technology.不过未来还有许多工作要做到。

接下来，在Cloudera的反对下，卡廷要开始专心于研究与大数据技术设施的法律政策。“Now that we have this technology and so much digitization of just about every aspect of commerce and government and we have these tools to process all this digital data, we need to make sure we’re using it in ways we think are in the interests of society,” he said. “In many ways, the policy needs to catch up with the technology.卡廷说道：“现在我们有了这项技术，商业和政府的方方面面完全都早已大幅度数字化了，我们也有处置所有这些数据的工具。我们现在必须确保用于它们是出于教化社会的目的。

从许多方面看，政策都必须紧随技术的脚步。”“One way or other, we are going to end up with laws. We want them to be the right ones.”“不管怎样，我们最后都要牵涉到法律。我们期望它们用在不顾一切的地方。

本文来源：j9九游真人游戏第一平台-www.makkyama.com