Smart Software Strategies: SPLASH 2015 Workshop Report

Workshop mission

What are the main issues that we discussed?

• software safety and reliability
• security issues, privacy issues
• developer responsibilities

Workshop raw notes

The complete workshop "raw notes" are available as a text file -- see http://manclswx.com/workshops/splash15/workshop_raw_notes.txt.

Workshop initial discussion

We had many questions in the initial discussion:

• how do we train developers to build reliable software for complex environments?
• what product quality issues are being ignored today?
• will certification and standards help improve reliability?
• is "innovation" in the smart technology domains creating problems with product stability?
• what kind of "quality assurance" is appropriate?
• are "agile approaches" appropriate for delivering new critical software services?
• how will society deal with liability issues when software components fail?
• are there good techniques to promote sharing information about product failures and performing good "root cause analysis" of common problems?
• what software and hardware do we trust?

In the workshop, we explored four major questions:

1. What is the "core knowledge" that software professions need today?
   • need "soft skills", not just coding
   • critical thinking skills; influence skills
   • yes, tech skills are also valuable: programming / languages / frameworks / OS / testing
2. How should we change the teaching process to train good developers?
   • cross-pollination between industry and academia
   • practical technology transfer techniques
3. What is the potential impact of Internet of Things?
   • conflicts about data ownership; designs with dangerous/risky assumptions
   • upgrade issues; "element management" for internet devices
4. How can we promote information sharing and effective root cause analysis?
   • create incentives to share failure data
   • use the aviation industry as a model -- Aviation Safety Reporting System (ASRS)

Issue 1: Core knowledge for software professionals

The discussion had a heavy emphasis on soft skills

• working with customers and teammates
• making decisions / giving good feedback
• critical thinking

Also important are "systems thinking" knowledge and techniques:

• building "systems" instead of just "programs"
• problem solving / using best practices

The top knowledge areas for doing effective development of complex distributed systems are:

1. problem solving; discrete math; abstraction (Note: "discrete math" is a common university mathematics class that combines probability, combinatorics, graph theory, game theory, and other similar post-algebra math training -- not including calculus, but very useful for doing work in software analysis and design)
2. needs analysis; dealing with customers; team management [key soft skills for product development]
3. ethics
4. economic tradeoffs in design and development
5. knowledge of standards (promoting interoperability)
6. best practices in design and development
7. critical thinking
8. influence skills (such as Robert Cialdini's book Influence: Science and Practice and Mary Lynn Manns and Linda Rising's book Fearless Change)
9. programming / languages; frameworks; OSs; testing; data structures

Issue 2: How to change the teaching process

1. Search is becoming an important way to acquire knowledge, but there are pitfalls
   • search requires some experience to judge the results
   • search fosters a "lazy shortcut" approach to learning -- you aren't forwarding your "understanding" of the problem
2. Training approaches should incorporate "building" as an activity -- it is good to "talk through the construction process" with a mentor or coach (to reinforce a good approach)
3. Cross-pollination is important in education -- collaboration between industry and academia to build experience and theoretical knowledge; it is good for academics to do some industry work
4. Tech transfer is a hard process -- there is a lot of knowledge in the heads of the original developers and users of a component -- how can this information be conveyed to the next generation of developers and users?
   • Don't take shortcuts -- make sure that there has been enough attention on "learning the lessons from the original developers and users".

Issue 3: Internet of Things

Internet of Things -- it is a technology for more than just toy applications:

• Networked devices and applications will sometimes be used in the control of potentially dangerous things
• Example: "smart home" applications such as home security and home control
• Example: devices used to assist elderly or handicapped people
• We have to raise our expectations for quality, reliability, and security of these products
• When do we need quality to be equal to aviation and medical systems?

We identified a large number of "risks" of poor design and poor quality in Internet of Things components and services. Here are the top four items:

1. Data Ownership is a big issue -- this may become an obstacle to widespread adoption of the technologies
   • Who wants to use components and services that spy on their users?
2. Dangerous assumptions, unsafe development shortcuts -- these may hinder use or reuse of software; how can we learn about *all* of the interactions and dependencies
   • Some systems are impossible to test -- we can't find all of the scenarios that are potentially unsafe or insecure
3. How to deal with devices that wear out and software that needs to be updated
   • note that cell phone software makers are not doing an adequate job of delivering software updates today -- Internet of Things will be an even bigger problem
4. Users should be able to view the status and activity of their devices and services
   • to learn what data is being shared -- some kind of "communications log"
   • we need this information to see who is trying to access or control our devices
   • without this information, how can you "trust" a smart device?

Issue 4: Reporting failures and issues; performing root cause analysis

It won't be easy to collect data from IoT vendors and customers, but there might be a good model to follow from the aviation industry: ASRS (Aviation Safety Reporting System) - asrs.arc.nasa.gov

• Everyone reports safety issues
• Data is used to perform root cause analysis
• And the results have been very useful -- FAA uses it to improve pilot training and airport operations
• Anonymous reports -- NASA will not tell FAA who is reporting (except for criminal investigations)
• If you report -- there will be immunity from civil penalties

Could we institute a similar program for the "ecosystem" of Internet of Things components and services?

• this might be valuable for identifying risks and quality problems
• much more efficient than legal action -- arguments over which software developer is "liable" for the failure
• the ability to report problems anonymously is really important
• *and* the commitment to do "root cause analysis" on serious problems is also important