Volume 17, Number 21 -- May 27, 2008

Evans Data Ranks Integrated Development Environments

Published: May 27, 2008

by Timothy Prickett Morgan

Farmers have their tractors. Managers have their PowerPoint and Excel programs. Plumbers have their wrenches and torches. And programmers have something given the utterly inelegant name of the integrated development environment, which just goes to show you that the marketing people get to name things far too often. Call it what you will, but the IDE is the tool that most of the professional programmers in the world--and there are millions and millions of them--stare at each day as they craft the applications that let the rest of us do our work.

Picking the right IDE is not an easy task, presumably, and programmers have their own preferences and prejudices regarding programming languages--just like the rest of us do for the important tools in our work lives--so it should come as no surprise that they would have strong feelings regarding the IDE that integrates and consolidates the programming tools that they use in their work. To help give programmers a sense of what's hot and what's not in the IDE arena, Evans Data, the consulting company that focuses on all manner of issues relating to programming and programmers, has released a study of IDEs that is based on a survey of 1,200 developers around the world that had users of IDEs from seven key suppliers rate them based on 15 different metrics, including ease of use, compiler performance, performance of resulting apps, quality of tech support, documentation, debugger, editor, and application modeling tools; programmers were also asked to give each attribute a relative importance ranking, too, and the whole batch of data was run through the statistical equivalent of a food processor to rank the IDEs relative to each other. Evans Data ran the survey behind the study in April in English, French, German, Portuguese, and Chinese, which is a pretty good sampling of the languages spoken by the programmers of the world, but in all fairness, Spanish, Japanese, and several of the languages spoken in India are missing from the mix.

The resulting study, which you can read here, compiled enough data to be statistically significant for Adobe Creative Suite/Macromedia Studio, Codegear Delphi, Genuitec MyEclipse, IBM Rapid Application Developer, Microsoft Visual Studio, Oracle JDeveloper, and the Sun Microsystems sponsored, open source NetBeans as well as Sun Studio IDEs.

After paying $2.1 billion to acquire Rational in December 2002, you can bet that IBM is as pleased as punch to find out that its Rational Application Developer got the best scores from the users of this IDE who took the Evans Data survey, beating out JDeveloper, Visual Studio, and Sun Studio by a couple of points. "It's safe to say that RAD's users love their IDE and tools as well as the support and care they get from IBM," the report states. "All of the IDEs in this survey are popular and very good tools, but RAD is just head and shoulders above them when it comes to user satisfaction."

That seems to be a bit of a stretch if you look at the data in the report, which I suggest you do. Rational Application Developer had what looks like a composite score of 168, with JDeveloper, Visual Studio, and Sun Studio clustering around 165, Adobe Creative suite at around 163, Codegear Delphi at around 162, NetBeans around 160, and Genuitec MyEclipse at around 154. The report doesn't say what the scale was for these, but this seems like pretty tight clustering as far as I am concerned. Across the 15 categories, developers who use Rational Application Developer ranked it with a high of near 200 points for the debugger to a low of around 140 on the integration of third party tools. What is clear from the charts in the report--and what is really useful for programmers who are looking at adopting a new IDE, or an IDE for the first time, which many are, is that IBM's ratings across the 15 categories are more tightly clustered than others, raising its composite score. The MyEclipse tool, by comparison, rates almost as high as the Rational IDE in terms of its editor and debugger (pushing 200 points), but some features only rated at just north of 100 points, dragging the overall composite score down. This happened even more dramatically for the Delphi IDE, which rated very low for Web design tools and its profiler, but rated much higher than RAD when it came to ease of use, compiler performance, and the performance of the applications it creates.

JDevelopers' bar chart looks almost the same as the one for RAD, and that explains why JDeveloper actually edged out RAD in the 2007 survey and the two were neck-and-neck in the 2006 survey. Visual Studio and Sun Studio are hard to distinguish from RAD and JDeveloper, but each one has slightly different data for each of the 15 categories.

While this comparison of IDE's has some value, each IDE was rated by its own users and therefore not as a true comparison and contrast. If Evans Data wanted to do something interesting, it would have a team of programmers actually rate the IDE's they don't know in addition to the ones they do know. Actually pay people to make a comparison and shake out the true differences, as seen through the eyes of experts, between the different IDEs available. Just because one group of programmers are happy with their tools in certain ways does not mean they would not be happier using other tools if they even knew what made those tools different or better.

Still, in the absence of any better data--and Evans Data is one of the few places where you can get any sense of what is going on in terms of market share and programming practices--this annual IDE study will have to do for now.


Developers Cool to Vista, Evans Study Finds

European Developers Embrace C#, AJAX

Java Is Catching Up to .NET for SOA Deployments

Developer Population to Grow to Nearly 19 Million by 2010

SOAs Are Being Implemented at Large Companies, Evans Data Says

Evans Data Cases Programming Language Popularity

Outsourcing, Offshoring on the Rise in North America, Evans Data Survey Says

Evans Data Poll Says J2EE Hits Tipping Point in the SMB Space

                     Post this story to
               Post this story to Digg
    Post this story to Slashdot

Sponsored By

Have a taste of System i modernization on us!

Simply pick 5 of your application screens
Seagull Software will create
a new GUI or web service for you
at no charge

Click here to get started today!

Editor: Timothy Prickett Morgan
Contributing Editors: Dan Burger, Joe Hertvik, Brian Kelly, Shannon O'Donnell,
Mary Lou Roberts, Victor Rozek, Kevin Vandever, Hesh Wiener, Alex Woodie
Publisher and Advertising Director: Jenny Thomas
Advertising Sales Representative: Kim Reed
Contact the Editors: To contact anyone on the IT Jungle Team
Go to our contacts page and send us a message.

Sponsored Links

COMMON:  Join us at the Focus 2008 workshop conference, October 5 - 8, in San Francisco, California
Help/Systems:  Explore operations automation and BI, June 17 - 20, 2008, Minneapolis, MN
Vision Solutions:  System i Management Tips Blog - Free i5/OS Tips Each Week!



IT Jungle Store Top Book Picks

Easy Steps to Internet Programming for AS/400, iSeries, and System i: List Price, $49.95
Getting Started with PHP for i5/OS: List Price, $59.95
The System i RPG & RPG IV Tutorial and Lab Exercises: List Price, $59.95
The System i Pocket RPG & RPG IV Guide: List Price, $69.95
The iSeries Pocket Database Guide: List Price, $59.00
The iSeries Pocket Developers' Guide: List Price, $59.00
The iSeries Pocket SQL Guide: List Price, $59.00
The iSeries Pocket Query Guide: List Price, $49.00
The iSeries Pocket WebFacing Primer: List Price, $39.00
Migrating to WebSphere Express for iSeries: List Price, $49.00
iSeries Express Web Implementer's Guide: List Price, $59.00
Getting Started with WebSphere Development Studio for iSeries: List Price, $79.95
Getting Started With WebSphere Development Studio Client for iSeries: List Price, $89.00
Getting Started with WebSphere Express for iSeries: List Price, $49.00
WebFacing Application Design and Development Guide: List Price, $55.00
Can the AS/400 Survive IBM?: List Price, $49.00
The All-Everything Machine: List Price, $29.95
Chip Wars: List Price, $29.95

The Linux Beacon
NYSE Euronext Trades Mainframes and Unix for Linux and X64

Canonical Founder Calls for Synchronized Linux Releases

AMD Ships Low-Power Barcelonas as Two More Execs Exit

New and Updated Barcelona Boxes Debut from Sun

VMware Tweaks Virtualization Stack, Boasts of Greenness and Sales

Four Hundred Stuff
Symantec Combats Phishing with New Services Offering

BCD Slings a New C#-Based GUI with Catapult 7.0

SkyView and Innovatum Formalize Partnership with New Product

Profound Eliminates OLTP Requirement with Web Enablement Software

140 Apps and (Hopefully) Counting for i 6.1

Big Iron
NYSE Euronext Trades Mainframes and Unix for Linux and X64

Top Mainframe Stories From Around the Web

Chats, Webinars, Seminars, Shows, and Other Happenings

Four Hundred Guru
Use PCOMM Scripts to Dynamically Build a Spreadsheet, Part 2

Use SQL to Strip Out Tab Characters

Admin Alert: Monitoring the Monitors

System i PTF Guide
May 17, 2008: Volume 10, Number 20

May 10, 2008: Volume 10, Number 19

May 3, 2008: Volume 10, Number 18

April 26, 2008: Volume 10, Number 17

April 19, 2008: Volume 10, Number 16

April 12, 2008: Volume 10, Number 15

The Windows Observer
Micro-Hoo is Back On the Table, But In a Different Form

Developers Cool to Vista, Evans Study Finds

Global Sales Save HP's Financial Cookies in the Second Quarter

Symantec Combats Phishing with New Services Offering

Microsoft Heads Aberdeen's List of Top 100 Tech Companies

The Unix Guardian
Global Sales Save HP's Financial Cookies in the Second Quarter

NYSE Euronext Trades Mainframes and Unix for Linux and X64

Sun Updates VirtualBox with Native Solaris Support

HP Ships Insight Dynamics for Managing Physical and Virtual Machines

A Word Cloud of IBM Server Brand Names

Four Hundred Monitor
Four Hundred Monitor's
Full iSeries Events Calendar


Seagull Software

Printer Friendly Version

The Way IBM Sees New Versus Prior i Platforms

The Server Biz Enjoys the X64 Upgrade Cycle in Q1

Evans Data Ranks Integrated Development Environments

As I See It: The Programmer as Artist

Reseller Mainline to Acquire Competitor Cornerstone

But Wait, There's More:

COMMON Belgium Shifts Focus from i to IBM SMB . . . That Windows-on-Power Rumor Surfaces Again . . . CDW Survey Says IT Shouldn't Wear Green on Its Sleeves . . . Magic Software Boosts Sales, But Profits Under Pressure in Q1 . . . The IT Services Business Keeps On A-Growing . . .

The Four Hundred


Subscription Information:
You can unsubscribe, change your email address, or sign up for any of IT Jungle's free e-newsletters through our Web site at

Copyright © 1996-2008 Guild Companies, Inc. All Rights Reserved.
Guild Companies, Inc., 50 Park Terrace East, Suite 8F, New York, NY 10034

Privacy Statement