Will current trends in DevOps and automation affect the role SDETs play?
If you want a definition of an SDET, see this posting. (As of 9/2/19, Wikipedia has no definition of the job title SDET.) If you Google the phrase "devops venn diagram" for images you will not see pictures of merely two overlapping circles with one circle representing "development" and another representing "operations." You will see DevOps in the middle of an intersection of three circles -- one of the circles is also QA! Moreover we may see a fourth circle for security in future Venn diagrams for DevOps; that is we may see DevOps being at the intersection of four circles symbolizing development, operations, quality assurance and security. Some people think that DevOps with security will remain its own specialized, combinational focus; to learn more about DevOps and security, see this article.
The DevOps movement is about combinations of specialties or getting away from specialization in favor of generalization; it is also about automation. For many of us, DevOps is about automating what you can program and programming what you can automate. There are benefits and limitations to automation. Humans can do non-systematic things that computers cannot do. In Nietzsche's Twilight of the Idols, the philosopher says "I distrust all systematizers, and avoid them. The will to a system, shows a lack of honesty." SDETs of tomorrow will still do what Quality Assurance Engineers did yesterday: manually find bugs that must be fixed. Bill Gates said many years ago that there are two rules of automation:
'1. “The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency.
2. The second is that automation applied to an inefficient operation will magnify the inefficiency."’ (Capgemini 2015)
Release automation is very powerful. Reliance on automation may grow, but human professionals will remain crucial. While the book How We Test Software at Microsoft was published before DevOps became a common word in the I.T. world, it is interesting that it said this: "Testers who understand programming concepts and computer architecture typically have the analysis skills that testing requires" (page 23). The requirements for a normal software tester are not necessarily high. In fact, being enthusiastic about testing may be sufficient. How We Test Software at Microsoft says "[t]he concept of hiring a software engineer with a passion for testing is powerful and is the biggest differentiator between the Microsoft approach to software testing and the typical industry approach" (page 23). It is important to note that prestigious hedge funds have followed Microsoft's lead in leveraging SDETs in their critical operations. In 2015 it was observed that the SDET role was becoming more popular due to Microsoft itself according to Ness.com. (The Ness.com article link used to be https://www.ness.com/significance-of-a-software-development-engineer-in-test-sdet-in-the-digital-economy/).
Some people think that "genius is as common as dirt" (taken from Weapons of Mass Instruction). Open source contributions come not just from institutions, but from eager job seekers doing spikes or from everyday computer enthusiasts. Sheer interest in interdisciplinary subjects and contextual knowledge about an environment may be more important than formal training. Can structured classroom work keep pace with the rate that technologies evolve? How We Test Software at Microsoft says that Microsoft did not hire "subject matter experts (SMEs) as testers" (page 23). Thus the DevOps trend toward generalization may trace some of its roots to Microsoft's aversion toward SMEs. Passionate individuals may not be content to specialize in one area thus to satiate their desire study multiple fields. Corporate executives in the I.T. industry should take notice that these passionate people may be the biggest contributors to the experimental and bleeding edge side of the DevOps movement.
According to a 2015 study co-sponsored by HP and others, DevOps will become a driver of testing. This finding, taken in combination with the 2008 How We Test Software at Microsoft, proves that DevOps engineers with a passion for discovering problems, would make excellent SDETs. Given DevOps engineers knowledge of architecture and programming, their penchant for integrating disciplines while using automation, a test-oriented mentality could make them phenomenally productive. Knowledge of network and operating system environments can be helpful for SDETs writing automation to ensure that a program is robust with the underlying platforms.
InformationWeek indicates, in their article published on 11/9/17, that businesses should spend more on software testing and possibly less on DevOps. The article found that DevOps has disadvantages if not implemented correctly. The article clearly indicates that a human running tests can address the limitations of automated testing. Spotify perceives testing to be a creative act and chooses to employ people to do manual testing (TechBeacon).
DevOps.com published an article in 2016 saying that "DevOps Does Not Need QA." They may be somewhat biased toward automation and overly optimistic about DevOps jobs. This website is primarily dedicated to technical DevOps topics (such as configuration management tools, systems administration, programming, and infrastructure automation). The Economic Times' CIO website recently published an article saying that DevOps groups are preferable to QA groups to transform a business toward a more agile state.
QA groups in 2017 are not necessarily composed of many SDETs. Cigniti.com's blog says that "by early 2020, almost all testers will need to wear an SDET hat to be successful in the field of Test Automation, that is going to become mainstream." A TechBeacon article, which seems to have been published in 2016, says that developers are now writing their own tests. This same article indicates that "test professionals" are better at writing performance test automation than regular developers. This admiration of specialization in testing contrasts with Microsoft's 2008 view that passion is key for obviating the need for subject matter experts (pages 23 and 24 of How We Test Software at Microsoft). It is unclear if Microsoft has revised their opinion, but it could be that Microsoft knows better than TechBeacon as the term SDET came from them.
In 2016 we started to see the emergence of the term "QA in Production" (SlideShare.net and MartinFowler.com). Spotify finds that it is cost-effective to do testing in its production environment (TechBeacon). Releases must happen rapidly in today's business world. Monitoring has become more sophisticated, and quality is important given the amount of competition software companies have.
Software consultant Viktor Farcic's view is that a siloed or "specialized" DevOps engineering department defeats the purposes of DevOps culture (technologyconversations.com). Despite this, dedicated DevOps teams do exist. In fact "[s]killed SDETs also often operate as part of a DevOps team in various capacities" (Aim Consulting). Farcic may be a visionary when he says that "[t]he level of collaboration between different professionals is directly disproportional to the number of barriers between them" (technologyconversations.com). The integration of professionals from multiple disciplines and backgrounds such as developers, fault-finding manual testers, skilled SDETs with refined knowledge in building automation tools, systems administrators who are knowledgeable of operating systems and where to find logs for root cause analysis can lead to the most comprehensive testing possible.
Facebook relies on automation for testing without manual testers (TechBeacon). "Both [Facebook and Google] companies emphasize personal responsibility (with Facebook shaming engineers that commit bad code), and see testing as part of the culture, not just a department somewhere in the building that has to clean up after the developers" (Business 2 Community). We know as collaboration across functional experts grows, SDETs often contribute to product development with code and design features for the software that they test. Cigniti.com's blog says that corporate Test Centers of Excellence (sophisticated QA departments) will be phased out because "[t]est automation developers today are now a part of the agile teams." The blog also says "that software testing budgets will continue to grow." The SDET role has been adopted by many companies that are exemplary in the DevOps movement. DevOps is about bringing development closer to a second function (operations). Thus the merging of development and QA produces the relatively new SDET role that is not siloed in a QA department but integrated with developers.
Job titles and professional focus may not lend themselves to exhibits at The Computer History Museum. There is little doubt that the modern day DevOps movement with the emergence of SDETs and relatively ambitious automation goals will make interesting history for what follows. We may look back at 2017 as an "early DevOps era" before many DevOps professionals transitioned into SDETs and the rest matured professionally into "DevOps 2.0 engineers." They may all work in one large group without departmental confinement achieving productivity we could barely imagine. The future may be that most I.T. professions will merge into one job title that did not exist in 2017.
The era hereafter may be without a scarcity of goods due to artificial intelligence. There are already predictions that learning algorithms can do the testing reliably and comprehensively (TechBeacon). Will automation with machine learning make us free from manual testing of software as well as the manual development of software?