SPLASH Workshop - Escaped From the Lab: Raw Notes
October 22, 2017
Vancouver BC, Canada
Escaped From the Lab
SPLASH 2017 workshop
One thing we try to do in developing software professionals--
- we want to help students to get into a career
- more than just "learning skills"
- we need to help them to be "good software people"
Many innovations today are coming from "outside" of the lab
- because of corporate cutbacks
- because of "targeting research" to particular political goals of the management
One interesting idea of "goals of software professionals"
(from the book "Grit: The Power of Passion and Perseverence" by Angela Duckworth)... three different levels of commitment:
- looking for a job
- looking for a career
- looking for a "calling"
Companies focus on intellectual property issues
- for example, innovation by acquisition
- (buying specific things they need from another company:
- innovations (patents and tools), people (experts in a specific area), or a market channel (a set of customers who need some products)
Social networks have formed a new fabric for how ideas/innovations spread
(more people are "influencers" in the world of innovation - more folks qualify as "tech transfer agents" today)
Over time, there has been in a reduction in having "managing innovation" be a special company role
- that is, there is no "director of innovation" anymore - just a bunch of innovators
Open source - there is a dark side:
- before open source, people would try to develop something with the hope of a monetary reward
- now, our profession is "gluing together" existing assets
This creates a lot of laziness (developers going with ready-made solutions, spending less time/effort thinking about real innovation)
(Note: It isn't all "cut and paste reuse". There are some areas where many major companies have active research programs, such as machine learning and quantum computing.)
Microservices is one big problem area for software engineering
- Although the individual microservices might be well-designed...
- there is a problem at the "orchestration layer" -- configuring the components to work together
- It can be really hard to apply good software engineering practices to the scripts and configuration screens in the orchestration layer.
What happens when you release a new innovation as open source?
- It makes you feel good
- You the inventor get some recognition
- But it can also "split the field" (a part of the user community will use your product, even if it isn't really the best for them)
- Would it be better for the inventor to work alone on the product, then prove to a big company that they should acquire and sell the new innovation?
Here is a possible example (not sure if this is a good case):
- Kotlin - an offshoot of Java with a lot of syntactic sugar
- It has been promoted for Android development
- Is it an innovation??
- Maybe it can be a replacement for Java, or maybe a replacement for Scala?
- Is it just splitting the development world for the embedded phone app development?
One more tangential idea: Static analysis tools for new languages
- This can be a positive thing: to help developers understand the intent of some code
- It can be a challenge when a new programming language is constantly evolving... to keep the static analysis tools up to date
DevOps: is this an innovation, or does it just bring chaos to development?
- No simple answer
- There is a lot of value in having a fast and efficient develop/test/deploy cycle - say 30 minutes or less
- Really useful in a world where you want new functionality to "go live" quickly
- A good thing about DevOps: more of the mundane work is automated - this gives developers more capacity for innovation
- but DevOps can be a nightmare with legacy software (because the code base wasn't designed to be tested easily)
It is good for a new language to have corporate sponsorship
- (company commits to using it for new development)
In many new development projects, you are building the foundation of a system that needs to live for a long time
- company commitment to a language for many projects will support that long-lived foundation
- and you can make investments in related tools and frameworks (with more confidence)
One big company commitment can spill over into other companies' work:
- For example, Facebook made a big commitment to Casandra database 10-12 years ago
- Other companies followed the trend
Another situation - when a company depends on an internal package that is *not* in their core business...
Big environments -- such as AWS (Amazon Web Services) can be really good, by providing many services useful in prototyping and early product development
- downside: you don't know where your data is located
- downside: vendor lock-in - this can result in high expenses in the long term
What is the best way to introduce a new programming language to a company or project?
- Using new language features for evolving application features
- which might be "reengineering" parts of the system that are changing rapidly
A lot of today's innovation is in the "web interfaces" (which are becoming increasingly convenient and sophisticated)
- They are using a combination of programming language innovations (such as scripting languages) and database / big data innovation
- Our "New User Experience" today is so much better than old text-based UIs
- the interface fixes spelling, offers best choices by "reading the user's mind"
- enables voice interface or multimedia
- But there are still possible pitfalls
- We shouldn't forget security, safety
- If our interfaces are too good, we might put people out of work
Some of the things that have changed in innovation. 30 years ago, we had this:
- roles: innovation manager, labs/researchers
- payback period -- it was longer than today... it could take years of building to make it work (how long to develop the first cellular telephony system?)
- budgets were big: this meant we had some slack... it allowed folks to do some flexibility and time to do some independent work (for example, there was more time to think about the design)
Now, we have a more "lean" approach (which can be good and bad)
- it drives down costs
- delivers ROI to company stockholders
- smaller increments of progress...
Ecosystems are more important today
- and innovation includes things like "better user experience"
- software pieces working together to serve customers' needs
We can think about some of today's big ecosystems: Android, Apple, AWS, Azure, GSuite (Google)
One issue: these ecosystems might sometimes *block* innovation
- because it is hard to move data between ecosystems
- (I might have a picture directory in GSuite that I want to move to the ICloud, but there is no good way to do a simple export/import)
- also, "expertise" isn't portable either
- (people are often experts in only one of these ecosystems)
In different ecosystems, experts use different terms/languages/vocabulary -- so it is harder to learn multiple ecosystems
Learning about ecosystems -- developing skills
- There is a "training versus education" issue
- Ecosystem vendors want you to be locked-in
- They prefer to offer specific focused training (on the specifics of their tools, services, and environments)
- Instead of offering higher-level "education" on concepts that might be useful in multiple ecosystems
This is a worrysome issue, because most detailed technical facts have a "half-life" of 18 months or so...
The economics of working with ecosystems:
- Employers don't want to pay more than they have to
- More developers are doing "cut-and-paste" development, using the Internet to search for the right code snippets
- sometimes called "stackoverflow.com" development
- This can lead to bad design - copying poor examples - especially code with security problems (which may go undetected in testing)
- There is an underappreciation for "code reading" and "code reviews"
- Especially in these cloud environments, more of the system is in the "configuration scripts" -- and the scripts may have much poorer quality than the code itself
- SAP - it's a nightmare, you need expensive consultants to make updates to the configuration scripts
- AWS - it is hard to move configuration information from one deployment to another
(This is true for most cloud environments - it's part of the lock-in. It is really a sign that the scripts are a kind of "highly coupled code".)
Another way to look at it: You are building an initial prototype, and you never leave the prototyping world...
Escaped From the Lab outline
Where do innovations come from?
- It used to be "research labs" (government, military, industry, university)
- Now: several places...
- university spinoffs
- driven by individuals (lone inventors)
- open source communities
- standards communities (for example, 5G wireless committees define many of the critical technology pieces and interfaces)
- business people (for innovations that are "business models" -- not just technical products in isolation)
- Accelerators and Incubators (including crowd funding)
- These are places where innovators make progress without facing real market forces directly
- Accelerators and Incubators provide funding, support, mentoring, sponsorship
- A special case is university environments... top business schools who help innovators make important business contacts... top tech schools (such as ETH) who help fill technical gaps in the innovations
- Integrating innovations into products and services
- option 1: build an internal product team (use your own staff from innovation to product release)
- option 2: acquire an innovation from another company (buy all or part of a company that has the innovation assets you need)
- option 3: release your innovation as an open source product - this can help build a community
- option 4: contests, grand challenges (get other people to compete for a prize -- they do the hard product integration work)
- Associated issues for innovation
- control of intellectual property (patents, copyrights, trademarks)
- motivations for innovation (including a list of the real world constraints that drive us to innovate):
- scarcity, regulatory changes and organizational edicts, time factors, inspiration from respected leaders
- money and position -- get rich or get tenure
- size and scope of innovations
- some innovations are small and incremental improvements
- some are revolutionary changes
- occasionally: porting old stuff to a new platform
- open ecosystem: oppportunities to work with more customers
- closed ecosystem: a way to protect your hold on your customer base
One characteristic of a closed ecosystem: high coupling, configuration scripts are not easy to move to another environment
How does this relate to programming languages and software engineering?
(Side note: there is a software engineering connection to some programming language evolution...)
- (But, we must interoperate with some old code -- code developed using an older language environment release.)
- (One idea: build some "verifiable transformation" tools to convert new language to old language...)
Some definition to distinguish "invention" from "innovation":
- Invention = creating something new and novel
- Innovation = creating something new that has value for society
Innovation will often create a new ecosystem (in order to be useful to users/customers) -- or it may "extend" an existing ecosystems
Here is an interesting set of ideas from a Huffington Post (2012) article:
- might be a big ecosystem (many tools, services, and other assets)
- might be much more modest
Once you escape from the lab.... how do you keep it going?
- In other words, can you get more users, customers, and revenues?
- Are there effective ways to "fertilize" invention & innovation?
Our 4 key dimensions for building invention/innovation:
- building funding and visibility
- getting feedback, building quality
- defending your innovation
- going for the big money
Here are some techniques for each of the four dimensions
- building funding and visibility
- incubators & accelerators
- venture capital funding / angel funding
- models for getting early users: freemium model / trusted tester model
- setting up training (and maybe certification in your technology) -- this may help you get users and buy-in
- conferences (show things off)
- get attention of the media, "influencers", and social networking ("build some buzz")
- crowd funding
- getting feedback, building quality
- offer payment for finding and fixing bugs (bounties)
- customer surveys
- A/B testing
- collect data from github (popularity of whose changes get "pulled" by the community)
- defending your innovation
- patents, copyrights, licensing
- speedy delivery (using fast delivery to deter competition) -- this is one area where DevOps can make a fast revision cycle possible
- government regulations
- going for the big money
- government funding
- moving your company to a "talent hub" (which will often be an area with good universities and opportunities for company/industry collaboration)
(Side issue: in a traditional "research lab", are we just doing research for the fun of it?)
How do we measure the Return on Investment (ROI) for our escaped-from-the-lab innovations?
- society impact
- ecosystem (especially if we achieve "sustained growth" of an ecosystem)
What is the role of partnerships in innovation?
- collaboration between lab staff
- and bringing in others (maybe some folks who are better at productization)
We might have "funded partnerships" or "unfunded partnerships:
A couple of ways ways to foster unfunded partnerships:
- company conferences or participation in trade shows
- open source collaboration
Funded partnerships are more complex... but there are different levels of gifts, contracts, and internships to make this happen most effectively:
- gifts/grants (company can get goodwill, mindshare on attacking certain problems, with a minimum of contract management overhead)
- contracts (traditional relationship for working across organization boundaries)
- consortium (managing complex relationships - achieving industrywide goals)
One positive side-effect of funded partnerships: talent migration
- it is a way to attract people to work for your company
- internships are especially effective...
One last question... what does this mean for a language expert?
The "lure of the cool"...
- you are working on enabling technology
- how do you make money? (or "get rewarded", or "grow in your field")
- do you go to more conferences? write more papers?
- get more people to use your stuff?
- and will they keep using it... or will they move on to something "cooler"?
Our goal as language innovators should be:
- helping people do more with less
It's a tool-building endeavor
Question: Why do companies develop (and deploy) new languages? (What is the value in creating a new ecosystem?)
- Apple developed Swift -- because Objective C is getting too old
- Functional languages -- can provide better performance, especially for parallel execution (and better verifiability...)
- Much of the world needs to do more work with data - especially "big data" - and we may want languages that make this work easier
- Some new languages try to give better support for fast build and unit testing (one good example is the Go language)
- Domain-specific scripting -- there are always new scripting language ideas for newer applications areas...
In our followup discussion, one person had several ideas to share -- some extra factors to consider in making innovation happen:
- It is essential for two sets of people to get together: the people who know the problems and the people who know the solutions
- This is one of the strengths of certain consortia and other innovation communities: such as MIT Media Lab
- Sometimes you have "mavericks who can't communicate" (need some facilitation that goes beyond standard productization mentoring, need to match them up with the right kind of business experts)
- Some "problem solving frameworks" might offer a way forward: TRIZ (a theory of inventive problem solving) is one relatively old framework (originally from 1946) --
- Another area to consider is the "diffusion of innovation" from Everett Rogers (originally from 1962) --
- The Cynefin framework by Dave Snowden (from 1999) discusses innovation in the context of complex systems theory --
- A more recent approach to approaching the issues of adoption of new programming language: Socio-PLT -- by Leo Meyerovich and Ariel Rabkin
-- see a 2013 InfoQ interview of Meyerovich:
-- also see a 2012 Google Tech Talk by Meyerovich:
Last modified: Oct. 23, 2017