The EAC’s Managing Election Technology Series

"The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency."

--Bill Gates

Overview

The Election Official of today is an Information Technology (IT) Manager – whether they think they are, whether they want to be, or whether they were trained to be.  IT Management requires a unique set of attitudes, knowledge and skills in order to plan, direct, and control contemporary election administration.  This series of guides to managing election technology identify the primary areas in which the effective Election Official must recognize their role as an IT manager and provides ideas and best practices to assist in  accommodating the demands of the modern election’s office.

Implementing Voting Systems with COTS Products

Over the course of the past several years, election jurisdictions seeking to purchase new election systems have begun a significant shift in focus away from the traditional approach of procuring predominantly proprietary election systems and toward procuring systems largely composed of commercial products. This shift follows what has become the practice over the past decade in state and U.S. Federal Government procurements, particularly those of the Department of Defense (DoD) and other large agencies. Many current Requests for Proposals (RFPs) being issued by government agencies now include a mandate concerning the amount of “COTS” (commercial off-the-shelf) products that must be included.

The considerations noted in this paper related to COTS products are cautionary: it is obvious to observers of election administration in this country that the growth in proprietary hardware/software costs are likely to continue, and that appropriate use of commercially- available products is one of the remedies that might enable election jurisdictions to acquire new and improved systems in a cost-effective manner. Additionally, COTS-solutions may provide needed flexibility in managing the service life of election systems as well as increasing their adaptability to changing requirements.

Like any solution to a problem, there are drawbacks as well as benefits to using COTS in election systems: while it may not be readily apparent to the election community as yet, many trade-offs exist when integrating a predominantly commercial product into an election system. Using COTS components in any given circumstance might prove cheaper and easier to implement, but this is certainly not careful analysis, reasoning, well-formed policy and practicable engineering decisions. COTS solutions present both opportunity and risks to jurisdictions who specify their inclusion in their election systems.

This paper discusses major considerations election officials should ponder before determining the extent to which their future election or voting system implements a predominantly COTS inclusive approach.

1.       Use the term “COTS” precisely.

It is important to get a clear definition of the term “COTS.”  The following definition is included in the EAC Voluntary Voting  System Guidelines, Version 1.1 and is the definition agreed to by the panel convened at the 2011 EAC Roundtable on COTS:

COTS: “Software, firmware, device or component that is used in the United States by many different people or organizations for many different applications other than certified voting systems and that is incorporated into the voting system with no manufacturer- or application-specific modification.”

Note that some items that appear to be COTS are really more correctly modified COTS1. Any change made by an integrator or third party to a COTS product disqualifies it from being truly COTS. True COTS should be able to be purchased at a retail store like Best Buy or directly from the manufacturer (Dell or Apple, for example) and be able to be plugged into the election system with no guaranteed. Election officials, in their capacity as IT managers, must understand that the use of a COTS component may be a reasonable solution, but its use should be the product of

1. Modified COTS (or MOTS – Modified Off-the-Shelf) refers to an off-the-shelf product that is customized by a commercial vendor to respond to specific requirements of the election community.

Likewise, it is equally important to define the term “proprietary”:

Proprietary:  “A technology or product that is owned and distributed exclusively by a single company and that is protected by patent, copyright or trademark.”

A common example of a proprietary product is Adobe Acrobat, whose Portable Document Format (PDF) files can only be read with the Acrobat Reader.

To further complicate best efforts to use these terms in a very precise manner, many products (and many, if not most, current voting system components) use numerous COTS or slightly modified COTS products) in what we generally term “proprietary” voting systems. This fact is best illustrated by the fact that several current voting system manufacturers use the same COTS scanning device as the basis for their proprietary precinct count optical scanning products.

2.       Understand the impact of COTS products on the RFP and selection process

The use of COTS products has significant impact on both the RFP requirements and the evaluation of proposals, an impact that must be understood early in the election system acquisition process. When purchasing COTS based systems, an election official must have a full accounting of the various implementations of the systems as well as who is responsible for each portion of the process supported by the systems. The purchasing jurisdiction must view itself as an election system integrator instead of the more traditional view of the election official as an election system implementer. The election official may want to include an IT integration requirement to the RFP so there is a clear line of responsibility and accountability for bringing together the various COTS based systems with already existing election technology in the office. In the end, the election official will be held accountable for all the components and systems working together as intended.

Election officials will do well to work closely with their IT procurement officers to ensure that there is no misunderstanding about the intended role of the COTS products, their maintenance, and the potential impact of unplanned and unauthorized substitutions.

3.       Understand COTS impact on system integration.

Integrating COTS components into your system is not necessarily simpler than integrating proprietary components, and may in fact be more difficult. The current thrust toward incorporating COTS products takes place in the context of “COTS-based voting/election systems.” That is, complex groupings of components, interacting in diverse ways, and in which introducing commercial components will simultaneously result in lower cost as well as providing the system “plug-and-play” characteristics

The phrase “plug-and-play” is significant, and is frequently the unspoken motivator for much of the current interest in COTS products. It conjures up a “software  centric” environment wherein heterogeneous hardware components can be easily inserted or replaced, and in which components interconnect and interoperate without modification or effort on the part  of the integrator2. It is based on the realities of the larger hardware world, where some degree of “plug and play” really does exist; boards, cables, printers, monitors, and keyboards can all be purchased, replaced, and upgraded independently and easily.

Even with this degree of interoperability however, changes in hardware can have significant impact on a system. (As an example, substituting a short, shielded cable with a standard 3m USB cable might very well have a negative effect on the EMI characteristics of the system.)

The “plug and play” notion also rests on assumptions about data that different tools will share. For the past several years, the IEEE P1622 committee3 has been working to specify a standard or set of standards for a common data format for election systems partially in order to assist in the integration and testing of components from different manufacturers and vendors. This work shows much promise, but still has an incredibly long way to go before allowing for true “plug and play” capability in election systems.

2. An integrator (sometimes known as systems integrator) is a person or company that specializes in bringing together component subsystems into a whole and ensuring that those subsystems function together. As noted above, the election administrator may well become the integrator of the future.
3. This work is now integrating with the EAC/NIST/TGDC VVSG standards development process.

4.       Understand COTS impact on the testing process.

Testing and validation of COTS-based voting and election systems is a substantially different process than testing and validating proprietary systems. All three pre- election system test efforts and post-election audits will be impacted by the introduction of primarily COTS based systems:

Federal Certification Testing

COTS products are, by nature, tested very differently than traditional voting system products. On the COTS manufacturer side, testing is market driven, not statutory or rule drive like Federal and State voting system testing. Significant research will need to be done to determine exactly what levels of testing are appropriate for COTS based voting and election systems.

Much will depend upon the availability of testing and quality assurance information available from the COTS vendor. Certainly and at minimum, COTS products will need to be functionally tested with the full voting/election system to be certain that at least the versions of COTS submitted for testing work as intended in the system. New requirements (VVSG) must be written to focus more on the product/system interface level and focus on performance or functional characteristics (i.e., what should the product/system be capable of doing as opposed to how the product/system should do it). These functional characteristics should also be quantifiable and testable by the VSTLs or others.

time-to market considerations win out over quality. In addition, while counterfeiting is an infrequent occurrence in COTS products, it does happen and when it occurs, it presents an additional obstacle to testing and assurance.

Federal certification testing also places significant importance on finding the root cause of failures encountered during the testing process. For some COTS-based system failures, finding the actual source of the failure, the root cause, or even re-creating the failure, will be more complex and time- consuming. This will be especially true when the COTS election system involves products from multiple COTS vendors.

State certification testing will have to address many of the same issues faced by the Federal certification program, but perhaps with fewer resources and less time to complete the testing.

States who engage in certification testing of voting systems will need to modify and validate their testing protocols to accommodate COTS components.

State/Local Acceptance Testing

The purpose of acceptance testing is to verify that the certified version of the election system is delivered to the jurisdiction and that the system can perform the functions required by the State and/or local election jurisdiction. Acceptance testing becomes a more complex process when integrating COTS components. Commercial COTS products typically have an accelerated development lifecycle and may change every 1-2 years or less. This will guarantee that units delivered to the jurisdiction will likely not be identical to either the certified version, or to versions delivered at other times.

Another factor to consider when acceptance testing COTS based systems is determining how to test to ensure that the COTS products work correctly with all other products in your dependent voting or broader election system. Do they negatively impact your election night reporting system? Your e- pollbook system?

Jurisdictions will need to implement strict version control protocols to ensure that all individual units are using the same software releases, or when not possible, to document the difference.

Logic & Accuracy (L&A) Testing

Pre and post-election logic and accuracy testing will be impacted as the system may actually introduce new COTS components at any time, thus making your system “new” again every single time it is used. To that end, election officials will likely want to revise L&A test scripts each time the system changes. These script changes may be major or minor, depending on the nature of the system change.

L&A Testing will continue to evolve so that not only is the ballot being tested, but that the system itself is being tested. L&A tests that utilize sampling methods will have to be expanded to include a comprehensive review of all equipment to be used in the upcoming election. This use model will need to include end-to-end testing methods, pushing past just vote capture and tabulation, to include election night reporting and post election activities that are dependent upon the correctness of the system.

Audits and Other Post-Election Testing

Any jurisdiction post-election audit procedures will need to be reviewed to identify controls that may be impacted by the revision of COTS components in the system.  In fact, the COTS components themselves may become the subject of such audits.

5.       Understand that a COTS approach makes a voting or election system dependent on the COTS vendors.

Vendor support for the commercial components in a COTS-based system is critical to the success of that system; many unforeseen problems can accompany a commercial system after deployment. The role of the COTS components’ vendors can be a decisive factor in successfully implementing and maintaining a COTS- based system. Several aspects are especially significant.

Does the COTS vendor supply adequate documentation for the component or components in question? Some COTS products offer extensive and useful documentation, but this is by no means universal. Is the documentation well- written and accurate? Can it be relied upon to be the integrator in order support election system testing, maintenance and future development? What kind of user support is available? Is the vendor responsive to user inquiries? In the current predominantly proprietary voting system world, problem calls will generally go to a help desk or to a field representative of your system manufacturer at their headquarters or in your own State or jurisdiction.  COTS products often use outsourced help desk/call centers that are located in various locations around the world with varying levels of service.

Finally, if the COTS component is to be part of a election system that is expected to operate for several years, what are the probabilities that the company will exist for that time? Finally, even assuming that the company exists, how long will it support the COTS product in question? Vendors often phase out their support for any given product: would such an occurrence have an impact on the maintenance of your election system?  By way of example, on September 30, 2015, Apple stopped signing4 iOS 8.4.1

4 signing is the process of digitally signing executables and scripts to confirm the and the first version of iOS 9 for the iPhone, iPad, and iPod touch, meaning users could no longer upgrade or downgrade to those versions of iOS . Apple is now signing iOS

9.0.1 and iOS 9.0.2 only.

6.       Realize the cost of maintaining a COTS system.

The presence of COTS components does not necessarily mean low maintenance cost.

COTS components can, in fact, cause complex problems during system upgrade and system maintenance. These problems could potentially exceed the maintenance cost of a proprietary system.

First, upgrading COTS-based system software means that as new releases of the commercial components are made by the various vendors, the system will incorporate them. In some instances, users can refuse upgrades and new releases, but it is safer to assume that the upgrades are inevitable.

Commercial vendors tend to support only a limited number of versions at any one time, and ignoring a vendor’s new releases is not an appropriate long term solution.

A further complicating factor is that different COTS components of the system will likely be upgraded at widely varying intervals; licenses will be invalidated and need to be revalidated for different parts of the system at random intervals. It should be kept in mind that COTS component upgrades can result in unforeseen problems including incompatible files and databases, different naming conventions, and software author and guarantee that the code has not been altered or corrupted since it was signed by use of a cryptographic hash.

introduction of new conflicts between COTS components. Depending on the number of COTS components and different COTS vendors, the effect of these multiple dependencies can vary from short-term user inconvenience to total system instability.

Finally, election officials will need to begin looking at voting and election system components more as disposable commodities rather than assets. “Maintenance” of these systems will most often entail replacing, upgrading and substituting components rather than the traditional model of sending a system back to the manufacturer, or having them come into your office or warehouse to conduct service or maintenance.

7.       Implementing COTS is not an automatic cost-saver

Although it might initially appear that the availability of COTS components brings down election system costs, there are also offsetting costs to consider and manage over the course of the COTS system lifecycle.

One of the most overlooked, yet potentially significant costs associated with the use of COTS products is effective COTS-specific planning and budgeting. Without adequate planning, the jurisdiction may become reactive to the inevitable COTS-driven obsolescence situations. These situations limit management options (when expanding jurisdictional options is likely one of the primary reasons for moving to COTS in the first place…) and could force jurisdictions to adopt more costly solutions when COTS components go end-of-life.