Windows Embedded Standard is a componentized form of Windows that enables the power of Windows to be adopted on embedded devices. The Windows Embedded Standard toolkit allows embedded developers to create custom images of Windows that are tailored to their needs. With these needs, the quality expectations of the toolkit and the customized images are t presented, and embedded developers using this product anticipate that it will be of high quality standards that meet the needs of their own customers. That’s why enforcing software product quality is taken to be a top priority for the Windows Embedded Standard’s development team. At this team, we follow the same standards and processes that the majority of the teams follow at Microsoft when developing their products, and this document will give a summary of some the main test areas that our product undergoes before releasing.
Test Area Guidelines
In talking about software quality assurance, you can find that many software testing areas exist. Some are functional while others are non-functional, and some examples of the non-functional areas would be the ones which are referred to as the “ilities” by the authors of the book “How We Test Software At Microsoft”, Alan Page, Ken Johnston and BJ Rollison (a few examples are dependability, reusability, testability ….). In the following sections, we present areas of software testing and briefly discuss it while giving pointers to where you can find more information on the topic. These guidelines are heavily used by the Windows Embedded Standard development team, and we recommend that they be followed accordingly and whenever possible by software development teams.
1.1. Accessibility
Accessibility is what empowers users to take full potential of their systems, and for some individuals is what makes the use of a computer possible. Microsoft has been a strong advocate of accessibility, and this is reflected in the vast support that Windows, and other products like office, have provided over the years.
Section 508 of the Rehabilitation Act was enacted to support the idea of having information technology available for people with disabilities, and to encourage the use of new technologies to accomplish this goal. To have a better idea on how Microsoft products addresses accessibility requirements from the Section 508 Standards, you can check out the Voluntary Product Accessibility Template (VPAT) which is a standardized form that shows how a product meet these standard regulations. Microsoft products VPATs are openly shared and can be found at: http://www.microsoft.com/industry/government/products/section508.aspx. Other regulations may apply depending on the product that is to be developed. For example, Section 255 of the Communications Act represents the “Telecommunication Access For People With Disabilities” regulation and should be also taken into consideration when designing a related product.
Although some regulations mandate that data be accessible by people with disabilities, usability is not. However, the general applications in industry shows that usability has become a best practice followed by many developers and tech companies. More information around accessibility can be found at Microsoft’s Accessibility website, and at the Microsoft Accessibility Developer Center.
1.2. Intellectual Property
Protecting intellectual property as well as avoiding related lawsuits is another issue that is to be considered when releasing software. Software patents, with their respective geographies, should be carefully considered by specialized individuals to make sure that the software product to be released is protected and that it doesn’t violate any existing intellectual property that might be owned by a different entity. License term documents provide a way to inform the user on what certain rights, restrictions, and obligations exist in using the software product. While different types of these documents exist, the choice of what type depends on the software product itself.
1.3. Security
Computer security has evolved to become one of the mostly written about topics in the computer software industry. Today, a lot of books can be found that talk about the processes that should be followed to ensure that a software product doesn’t compromise the security of a system that it is installed on. Since the early 1970s, malware such as “Creeper” and “Rabbit” were among the first to find their ways into many computers which led some computer developers to find ways to track and remove the malicious software. “Reaper”, which differed a lot from the antiviruses that exist today, was developed for the sole purpose of tracking and eliminating the “Creeper” worm. Today, both malware and anti-virus software have become so complex in their exploits and algorithms.
Software development cycles have progressed to include accountability for software security as early as in their planning and design phases. Software vulnerabilities range from buffer overflows to code injections and it is highly recommended that software developers are trained to write a more secure code. A simple input validation test might, for example, prevent a buffer overflow attack on a certain code block. Guidelines on how to write secure software, which may be language specific or not, widely exist and they should be followed by software developers to ensure that their code has a relatively high security level. An example guideline is the “Security Guidelines: ASP.NET 2.0” which can be found on Microsoft’s MSDN web site. A more general process guideline which Microsoft has, and which most of the other guidelines stem from, is called the “Security Development Lifecycle” (SDL). The SDL guideline, or a somewhat similar one, is highly recommended and is commonly adopted by many software developers and companies.
One commonly used process in software security is called “Threat Modeling”. The main purpose of this process is to identify software security risks and to and prioritize them according to their impact on the quality of the software product. Writing a secure software product is not an easy process, and what threat modeling does is that it provides the developers with a systematic way to identify the attack surfaces and to account for any risky and costly areas in the code they’re writing. Many approaches of threat modeling are out there. Attacker-centric, software-centric, and asset-centric are few examples on how this modeling process can be done depending on the product being developed, and it’s the development entity’s choice on which approach or combination of approaches to use when designing for security. A good book on this topic is a Microsoft Press book called “Threat Modeling” and is written by Frank Swiderski and Window Snyder.
Fuzz testing is yet one of the used techniques in software security testing. This semi-random testing technique tests how the code reacts to different randomly generated inputs. Examples of inputs would be negative numbers, unexpected data, or even data that might exceed the expect buffer sizes. Although this type of testing has a disadvantage that it randomly selects different data to pass to the code, it usually can detect a lot of bugs that are related to areas such as memory leaks and security threats.
1.4. Privacy
If you’re thinking that privacy assurance is a part of security testing then you are right! However, we , the writers, felt the urge for it to have a special attention in this document emphasizing the fact that privacy has grown to become a significant entity in releasing software. In the past decade, more and more online services have emerged which in their essence focus on the user with respect to personal information and preferences. With this rise in online services comes the significance of user privacy. Critical users’ information has unwantedly been acquired by hackers which have caused legal and business implications. Luckily, Microsoft publishes privacy guidelines which are part of the SDL. It is highly recommended that these guidelines are followed so that the legal risks are diminished and the end-user trust is increased. For example, if a software product or a service gathers user data, then a legal notice should explain to the user what data is being collected and the reasons behind it. Customer’s consent is one of the basic requirements in this case.
1.5. Geopolitical
With the Internet breakthrough, a software that has been developed anywhere in the world may be instantly accessible by any internet connected user around the world. Users with different languages and different cultures may end up using the same product and some phrases or words that might be non-offensive in one part of the world, may turn out to be very offensive in some other parts. When releasing software, it is critical that a scan is done on the included data to ensure that any wording cannot be mistakenly comprehended as offensive. Many software companies which develop their software in different languages do have advisors who are local to the area in which the software is to be published. Checking for sensitive data before releasing a software product helps in building a trustworthy image, lowering legal implications and even product bans and boycotts.
Sensitive data is not limited to offensive material (which might include phases, terms, icons, flags, etc. … ) only, some other types of data like credit card numbers, social security numbers, or even any internal company information that shouldn’t be disclosed to the outside world should be scanned for. There are some tools out there that can be used for the purpose of sensitive-data scanning, and most of them have special term database that they compare the data against and try to flag any suspicious data. A lot of false positives may be generated from such tools; however, some of them do have the ability to adapt and learn from their users which helps in minimizing the false positives numbers. A huge advantage of using such tools is the speed at which such tools can scan. An example of such tools is “Spider” which is an open-source tool that is developed by Cornell University.
1.6. Software Integrity
A wide range of technologies can be used to achieve software integrity in the product. As a first step towards that goal, it is the software development entity’s responsibility to ensure that the software is virus and malware free. Specialized 3rd party companies have come up with some of the best tools known for capturing any suspicious virus or malware that might be embedded into a software product, and using such tools provides more confidence that the product is safe. Another issue that comes to mind when talking about software integrity is the authentication. With the help of code signing and certificate generation, a software entity can provide a level of trust for its users worldwide. Microsoft provides the tools and guidelines on how to use code signing in a software product.
1.7. Powerful Tools
Many of the tools needed to accomplish the tasks described above are available as part of the Windows SDK, DDK or Visual Studio product suite, and below we describe a few of them.
Beginning with Visual Studio 2005, the compiler would issue C4996 warnings if any C runtime library functions were used that were known to be vulernable to buffer overruns (i.e. “banned” APIs). By strategically inserting a “#pragma warning (error: C4996)” in your header files, see below, you can pretty much guarantee that none of these functions are accidentally used in your code.
#include "stdafx.h"
#include "string.h"
#pragma warning (error : 4996)
int _tmain(int argc, _TCHAR* argv[])
{
char a[] = "A String";
char b[256];
strcpy(b, a);
return 0;
}
The compiler also allows for runtime buffer security checking with the “/GS” compiler switch. This can also be set via the UI on the project properties page (Code Generation).
Another example is that the “enterprise” versions of the C++ compiler support the “/analyze” switch. This switch enables the “PREFast” C++ code analyzer. For managed code applications, the FxCopy tool provides a similar functionality.
Conclusion
Meeting the customer’s needs in any software product is a big challenge. While different software products may have different set of quality requirements, having a set of standardized guidelines represents one of the key main points in tackling the problem of enforcing software quality. As it is said, “quality is never an accident” and attaining a high level of it can’t be successfully done without good execution strategy which goes alongside good planning.
-Sami, Jim and Jeff
![]()
