Question
What is an SDET and where did the term originate?
Answer
The book How We Test Software at Microsoft says that the formal title for a software tester at Microsoft is "Software Development Engineer in Test, or SDET" (page 23). This SDET acronym (free of punctuation) was adopted in 2005 (page 24 of the above referenced book). Microsoft's adoption of SDET gave this acronym recognition and serious meaning given the company's leadership in the industry. There seems to be no evidence that the term SDET was widespread before Microsoft employed the job title. The "T" stands for "Test" while the preposition "in" is not referred to in the acronym. The term is pronounced "ess-det," like the letter "s" as the first syllable and the word "debt" as the second syllable. The word "debt" rhymes with the word "pet" (as you can hear with this Microsoft Ireland video on Youtube).
How We Test Software at Microsoft says that a regular software engineer at Microsoft is called "Software Development Engineer, or SDE" (page 23). This book, written by Microsoft employees, says that "[t]he similarity in names of the two disciplines is by design because testers at Microsoft are developers" (page 23). So an SDET position was designed for an employee to develop code to test software. Whereas regular software engineers design code, SDETs specialize in finding bugs and using automation to test the software. Microsoft is still completely aware that not all testing can be automated. This book, How We Test Software at Microsoft, was published in December of 2008, but it still has relevance 10 years later.
Professionals who test software in a traditional method with manual procedures can be given the job title of one (but not limited to) of the following: Quality Assurance Analyst, Quality Assurance Engineer, Quality Control Analyst, or Quality Assurance Engineer. Quality Control focuses on production defects whereas Quality Assurance attempts to set guidelines for the development process (according to TechTarget). Traditional software testers (e.g., a QA Engineer) may design unit tests for atomic features of software. Both QA analysts and SDETs test software, but SDETs create automation to do it. SDETs may create a script or a comprehensive framework for much of the testing itself. SDETs may examine performance, automate tasks, or write complex programs to check reliability beyond manual testing of specific functions of software. SDETs often participate in the development of code whereas QA analysts might not (Software Testing Class). SDETs are more frequently part of the development team itself rather than being an employee of a separate QA department. SDETs are a hybrid between a QA analyst (a manual software tester) and a software engineer. The bottom of this article has links to other websites to define an SDET with greater details.
In Eric S. Raymond's book The Cathedral and the Bazaar (on page 33), he says that developers focus on the theoretical aspects of the code whereas testers focus on the pragmatic behavior of the software. We see that SDETs to some extent blend the best of both worlds. As of May 2020 we are not sure if SDETs with developer backgrounds are preferred in the industry's recruitment process over those with QA backgrounds.
DevOps engineers are thought of as hybrids of developers and operations analysts (or systems administrators). SDETs merge the disciplines of QA and development in very much the same way DevOps engineers merge the disciplines of development and operations. SDETs leverage automation in their daily work in the same way DevOps engineers leverage automation in their daily work. Wheres as a DevOps engineer may automate an operations task (the creation of a virtual server), an SDET will automate a series of tests to find bugs in a new version of software (write a Python script to run Selenium tests). For code that is deployed at a large scale (i.e., across thousands of servers), testing every unit in every environment is possible with automation; in the past testing may have been limited to a portion of an environment that was thought to be a representative sample.
SDETs may write test harnesses in code to do sophisticated quality assurance.** For example they may write code for observability via leveraging metaprogramming in the code that is being tested.***
For software that is unexpectedly valuable and has a useful lifespan longer than expected, the investment in testing automation can pay off greater than predicted. The book How We Test Software at Microsoft says that Microsoft never planned on supporting Windows 95 through the year 2005 (page 27). Some software has a useful life far longer than expected. Therefore SDETs can write automation scripts or frameworks that can be of more value than what their company initially estimated. Automation is key to putting software through larger numbers of tests too.
Gene Kim's IT Revolution website says that "[n]ow when any Google developer commits code, it is automatically run against a suite of hundreds of thousands of automated tests." Google uses the title SET (which stands for Software Engineer in Test) instead of the term SDET. Both Microsoft and Google believe in introducing new code or refactoring existing code purely for the sake of testability (page 67 of How We Test Software at Microsoft and Infoq.com).
If you search for "Software Engineer in Test" or "Software Development Engineer in Test" on Dice.com or Indeed.com, you will see that many companies have adopted the words of the SDET acronym as opposed to the Google equivalent or near-equivalent (the acronym SET). Fidelity Investments uses the term "Software Engineer in Test." It appears that IBM and HP do not use either of these job titles (as of November of 2017), but Apple seems to use both “Software Development Engineer in Test" as well as “Software Engineer in Test" (based on job searches for these two titles on each of these three companies’ websites). Amazon’s job site shows that they use SDET but use the word "Dev" as in “Software Dev Engineer in Test" for their version of the job title. SDETs can work in a variety of departments depending on the company. Evidently they initially worked in the Test department of Microsoft.
According to TechBeacon Microsoft employs approximately two SDETs for every three regular software engineers. According to the book How We Test Software at Microsoft, Microsoft employed one tester (either an SDET or a traditional, manual-testing professional) for every one regular software engineer (page 31). At Google there are far fewer SETs for every regular software engineer. This may have less to do with the prevailing culture and more to do with the products offered. Microsoft's products are mostly shippable and have to be compatible with different hardware. While Android needs to run on different hardware too, other than Chrome many of Google's products are SaaS offerings. Business 2 Community reports that "Google’s achieved this [having relatively small number of testers] by creating a culture of testing, and not an abstract, disconnected testing department." The lack of disconnection and mostly integrated aspect of testing is core to the DevOps movement itself.
The book How We Test Software at Microsoft indicates that security is an important part of testing along with "ilities" such as just reliability, maintainability, usability and portability (page 250). Having secure products helps build trust with consumers and employees of a high-tech company.
In most of years of their existence, Google grew with the motto "[d]on't be evil." They ostensibly still have a code (no pun intended) of conduct to act honorably. This niceness has helped their testing efforts too. Not many companies have the quality and quantity of software testing that Google has, especially with their staffing ratios of comparatively few Software Engineers in Test. IT Revolution reports "[w]hat enables this system to work at Google is engineering professionalism and a high-trust culture that assumes everyone wants to do a good job, as well as the ability to detect and correct issues quickly."
Google's institutional commitment toward quality software involves a blend of development and testing. They practice shift-left testing and have a corporate-wide testing culture (not reduced to a mere department or a practice confined to group) with ethical conduct for establishing trust among employees. Their SETs have helped create a software program called the Google Test Case Manager too. Perhaps the Google Test Case Manager will be made available to the public and make the same impact as other Google contributions have made.* To learn more read the book How Google Tests Software. Google is not alone in believing that trust is important for employees working more closely together (DevOps.com). Google was not the first to recommend testing should be incorporated in the initial development process; the book The Mythical Man Month (on page 69) says that testing should be done "early and simultaneously" in the design of software.
* White papers published from Google were the basis for Hadoop. Google open sourced Kubernetes (ACM).
** The book The Mythical Man-Month says "[m]ethods of designing programs so as to eliminate or at least illuminate side effects can have an immense payoff in maintenance costs."
*** "Metaprogramming is a technique of writing computer programs that can treat themselves as data, so they can introspect, generate, and/or modify itself while running." (This was taken from page 158 of Expert Python Programming. The newer 4th edition is available here.)
Additional Information and Citations
The book How We Test Software at Microsoft uses the term DNA to describe the attributes of an effective SDET (page 23). Google also uses the term DNA to describe the traits of an ideal SET and not SDET (GoogleBlog). DevOps.com has an article about how an SDET spends time on an average day.
To learn more about the contradistinction between an SDET, a software tester, and a software engineer, you may want to read these links.
(1) https://www.infoq.com/news/2011/03/Ensuring-Product-Quality-Google
(2) http://www.softwaretestingclass.com/what-is-difference-between-sdet-and-tester/
(3) https://softwareengineering.stackexchange.com/questions/191913/what-is-a-developer-in-test
(4) https://sqa.stackexchange.com/questions/25119/what-does-a-software-development-or-software-engineer-in-test-do
(5) https://sqa.stackexchange.com/questions/6408/what-is-the-difference-between-software-test-engineer-test-automation-engineer