What is wanted is domain knowledge and the technical expertise to produce it, to encode it, to communicate it. Watch Guardian editor David Leigh fumbling about, after having been granted the keys to WikiLeak’s Cablegate kingdom:
Leigh set off home, and successfully installed the PGP software. He typed in the lengthy password, and was gratified to be able to download a huge file from Assange’s temporary website. Then he realized it was zipped up — compressed using a format called 7z which he had never heard of, and couldn’t understand. He got back in his car and drove through the deserted London streets in the small hours, to Assange’s headquarters in Southwick Mews. Assange smiled a little pityingly, and unzipped it for him.[But now he could not] call up such a monolithic file on his latop and search through it in the normal simple-minded journalistic way, as a word processor document or something similar: it was just too big.
Harold Frayman, the Guardian’s technical expert, was there to rescue him. Before Leigh left town, he sawed the material into 87 chunks, each just about small enough to call up and read separately. Then he explained how Leigh could use a simple program called TextWrangler to search for key words or phrases through all the separate files simultaneously, and present the results in a user-friendly form. [source]
This passage produces pride in a geek—ah, those simple-minded journalists! Good thing we’re here to rescue them—but pride cometh before a fall. I worked for a while as the sole geek at a non-profit. I tried to teach people to use Google to find solutions for computer problems. One day I was trying to make coffee and wondered out loud how to use the percolator. Our dour accountant cracked, “Hey Chad, Google it!” Yuk, yuk.
With my current employer the domain is statistical analysis of public opinion. The more I can know about that, and the more the statisticians can know about software development, the better we’ll do. The standard software for statistics is R, which like so much in the sciences has decades of features and inertia but is below average from a usability and software engineering point of view. Thankfully we have people who “want to build a statistical computing environment that trounces R.” We shall see. In general though the need to master the means of production is being more widely acknowledged in the sciences:
[A]s computers and programming tools have grown more complex, scientists have hit a “steep learning curve”, says James Hack, director of the US National Center for Computational Sciences at Oak Ridge National Laboratory in Tennessee. “The level of effort and skills needed to keep up aren’t in the wheelhouse of the average scientist.” [source]
The middle class of musicians and filmmakers have already mastered their means of production, though at the high end the roles remain distinct. At the high end of software development we are told that “[e]ngineers handle entire feature[s] themselves — front end javascript, backend database code, and everything in between.” This advice is trickling out to the rest as well.
In general, the more of the stack you can fit in your head, the better. And the better the pieces of the stack, the more you can fit in your head. Where would David Leigh be without TextWrangler? Too bad nobody told him about Google, or he could have saved a trip across London in the small hours. And I could have made coffee.