SPLASH Workshop - Smart Software Strategies October 25, 2015 Pittsburgh PA Raw notes from the workshop ---- sheet 1 Smart Software Strategies workshop Safety - Nancy Leveson: can't add on safety later 5-nines reliability [reliability target in telecom] Software that "cheats" [Volkswagen diesel engine control software - detects when emissions tests are happening] Security - what security? [we don't do very well at preventing break-ins] How do we train people? (to write good smart software) Social media and privacy Laws and regulations Selective blindness - don't understand all of the constraints [or the "context" of our software systems] ---- sheet 2 Certification Programming Innovation [one way we break out of the old way] - you relax constraints when you are being innovative - or you are being asked to make changes (for higher speed, faster response, smaller size, less power consumption) - innovation is sometimes *driven* by new constraints (regulatory, new hardware, ...) Smart phone was an *evolution* Some smart things are aimted at a new domain / disruptive Military ofoten drives innovation - because they have unlimited money Y2K - driven by a date / the need was *urgent* - lots of legacy code that didn't consider the constraint ---- sheet 3 Internet of Things - a flood of interconnected devices - lots of autonomy - who is creating the software? and are they considering security/privacy issues? - can we have *innovation* and *stability*? Reuse of existing components - open source has issues - many eyeballs find good exploits Facebook seems to have no corporate QA?? (they are not posting QA job openings) - what are the conseqences of failure? - "reputational" - privacy violations * many agile companies say they don't need QA / not critical software * *or* my QA is built into the agile process How do I make sure I cover *all* of the code? ---- sheet 4 Smart technologies / components / micro-services - there is extra cost in testing * local scenarios * end-to-end scenarios - management may not understand the pitfalls of bad software - "you have to test thoroughly, or you will be out of business" - liability for failures * can we point the finger at some other companies? [we might claim that the problem is due to someone else's components] - liability at the source level - insurance for component providers (will insurance companies demand certain practices? -- will the mandated practices work? will we get superficial compliance?) Adequacy of testing - not feasible to test all combinations, all paths - practices can reduce errors, but not eliminate them - reduce risk ---- sheet 5 Who can you trust? - government? - programming experts? - compiler / tool chain? (compilers have bugs...) * (assume it has been hacked by spies, or kids) Privacy - should we publish the names of people who create bugs... (side note: in the California drought, one way to get people to comply with the water use regulations -- they are publishing the names of people who are wasting water...) How to change the teaching process? - maybe you have to fly the plane you design - the coding is the fun part, but doing the rest of the work [testing] seems tedious - Kent Beck said "big upfront design is bad" - maybe it is for him, but highly critical applications need more design - Some teachers and consultants... they like to claim that softwaer systems are the most complex human constructs... is this true? - Can agile work in all cultures? * Scrum with command and control (violates the spirit of Scrum, but you still might be strictly following the rules...) * Agile "embedded" in a waterfall ---- sheet 6 Kanban can be a good alternative approach when Agile won't work (following the Lean principles) * one size doesn't fit every problem Training - we focus training on "skills" (practical, but short-lived) Education - the focus is on concepts (problem: students might not be able to *do* anything -- there is a gap between theory and practice) ** we need both ** and we need "conceptual thinking" mindset (we need to be "computer scientists", not just "computer users") Security -- the technical part - encryption - authentication strategies ** many CS folks are interested in these things - but many key issues in security are "practices and procedures", and they aren't really *fun* - consultants and academics - often don't want to follow their own advice ---- sheet 7 Sharing information about defects - aviation failure reporting system * all problems are reported to a public database (anonymized) * incentive to report: airline is absolved from liability - Root cause analysis * example results: better training about runway procedures triggered by increased number of runway incursions In software development, we have a different culture of reporting problems (even internally) - because of delivery pressures Medical training - it is more of an "apprenticeship" model - less teamwork and sharing in the software world - longer term training in SW: people learn a lot on a masters thesis project - modern issues in education: teaching needs to be entertaining?? or at least not boring ---- sheet 8 Internet of Things - new devices - are we going to be ready? - will we have standardized infrastructure? - or will there be waves of *innovation*? (which may cause usability, interoperability, and security problems) - convergence on a small set of standards? - differentiation of produces pushes away from standards - it's too early for standards in some areas Risks of autonomous devices - software needs to be higher quality - emphasis on software safety (will this happen?) - reliability issues - privacy issues - how to manage updates (maybe use DevOps concepts and techniques) ---- sheet 9 Discussion topic list: - How to change the teaching process (5 votes) - Safety and security (2 votes) - Internet of Things (4 votes) - Privacy and Liability [regulatory things] (2 votes) - Sharing information - defect reporting / Root cause analysis (4 votes) - Certification and Licensing (0 votes) - Core knowledge (5 votes) - Cost of quality (0 votes) - What can we do to drive change? to get decision makers to take issues and failures seriously? (0 votes) - Components / micro-services (1 vote) ---- sheet 10 Topic 1: Core knowledge academic => core curriculum industry => knowledge to solve problems and do work 1. problem solving; discrete math; abstraction [discrete math is a non-calculus intermediate math course that covers probability, combinatorics, graph theory, game theory, and other topics that are very useful for software developers] 2. needs analysis; dealing with customers; team management [soft skills] - identifying the essential elements of a problem 3. ethics 4. economic tradeoffs - items 3 and 4 are important for building systems One problem in the academic world is the attitude of "I don't work with others - that would be plagarism" -- not a useful view in industry at all... teamwork is really important on critical problems Best practices from industry .... - unfortunately, academic conferences (such as ICSE) are downgrading the value of sharing industry experiences 5. Knowledge of standards - why is ignorance bad? - you reinvent the wheel if you don't know the standards - your innovation doesn't interoperate with existing components or technologies ---- sheet 11 6. Best practices - (by the way, best practices are not always a good way to work) - (because we might use them blindly without considering the context) - (and we try to reduce every problem to just one thing) 7. Critical thinking - games to develop/stimulate critical thinking 8. Influence skills (another soft skills item) => understanding politics - social proof; reward structures - see Cialdini's book - Influence, Science, and Practice - also Manns and Rising - Fearless Change - as a young professional, you have to "establish presence" (by always being at the key meetings) and "establish credibility" (by always being right / being prepared) 9. Programming/languages; frameworks; OSs; testing; data structurs [the traditional technical stuff...] - finding the right language/library/framework for the problem (also, developers need to have good *domain knowlege* to build a good system) ---- sheet 12 Topic 2: How to change the teaching process It is a topic that is outside of computer science Educators must be about more than just "passing the test" Future trend: Flipped courses - lectures are on-line recordings - class time is interactive: discussions, activities Search - is becoming a major method to acquire information [this was from the SATURN 2015 keynote by Mary Shaw: https://www.youtube.com/watch?v=S03bsjs2YnQ] - search is a skill - but you need context to search effectively - you might find things that are inaccurate or reflect a point of view [and you need to be able to identify when the information is untrustworthy]] - you may get really bad examples [the biggest hazard of using examples from a search of stackoverflow.com - you might copy someone else's bad code] Is search a "lazy shortcut"? Does it inhibit developing skills? - you take a "prepared answer" - it doesn't forward your understanding "Learning" sometimes requires getting to the edge of your understanding - you are primed to think and learn One useful technique in teaching a complex concept: - The instructor builds a "partial" design/program - The students fill in the missing parts In aviation training, there is a good practice to reinforce complex processes: when the flight instructor has the student pilot "talk through the maneuver" as he/she is doing it ---- sheet 13 Cross-pollination: more industry involvement in teaching in academia; get academics to work a while in industry - need more of a conversation... not just a tourist at a company - it is good to see the real problems - interactions at multiple levels Students: mentoring and internships (sometimes a summer is too short) People who teach - need to have some industry experience Where should teaching happen? - formal courses - design reviews; code walkthroughs - building something Tech transfer - it isn't easy -- a big topic - there is a lot of knowledge in a software component - in the heads of the original developers and users - how can a new person come up to speed? [one idea that has been used... transfer people around to build bridges] ---- sheet 14 Maybe you need a long tim eto learn software - like playing piano, medical training - working with assistance ("guided practice")... before you can work alone Need an investment in *learning* (and continuing learning) to be an expert practitioner - reload *skills*, not concepts You need to kow when to ask for help (and how and whom to ask) **How to answer questions on StackOverflow: this is something that might get you a job - we heard of one case where Google actively recruited someone with a high rating on StackOverflow Learn by doing: - projects - games - internships connected to curriculum - mentorships / teaching assistantships - labs ---- sheet 15 Topic 3: Internet of Things [brain dump of IoT risks and issues] Data ownership! (privacy is essential) Components built by the clueless... - need to plan the interactions - security and safety issues - unintentional feedbacks - cascade of messages (We aren't sure what all the interactions may be... could be more than we originally planned... they might change as the system evolves) (not all developers are trained to work in this area) (counter-intuitive results due to parallel composition -- need training in parallel design, asynchronous, ...) Customization versus stability (it's a tradeoff -- more ability to do customization within the architecture/design makes it harder to keep the system stable over a series of releases) ["Domain engineering" is a field that had some good ideas to handle change... ==> control variability, exploit commonality] Unwarranted and dangerous assumptions - things like "unsafe shortcuts" taken by developers when they are under schedule pressure Unexpected interactions and dependencies (and we tell ourselves "our build system will figure it out") ---- sheet 16 Controlling things from the Internet? (do we want to do this??) * that will never be secure! * but it only needs to be secure enough Child safety Devices can wear out (15 year-old garage door; 2 year-old cell phone) Upgrade issues (it's already a problem for iPads...) Manual override How can I see the *status* of a component or system? - what data is being shared? - telecom calls this an "element management system" * it is a standard part of managing telecom systems and other enterprise systems Can we see who is trying to "control" our device? (log files -- and make things simple and clear enough that a normal user can understand them) Discovery scenarios... components and services "find each other automatically - how do I register for services? - what about failure cases? Fundamental question: How do you trust? ---- sheet 17 How do you trust? - what are the scenario sto establish a trust relationship? - what are the possible consequences? How can you *untrust*? - how do you "unfriend" someone in the Internet of Things? Topic 4: Root cause analysis / Reporting It would be good to have central reporting of incidents - ASRS (Aviation Safety Reporting System) - run by NASA Need to have a group that analyzes the data [not enough just to collect data...] Use data to improve education - collected good practices Will the "rate of new product deployment" be too fast?.... difficult to keep up with lessons learned