Deciding on your test strategy isn’t always a straightforward decision. One of the factors that is likely to influence your decision is test ‘coverage’. Very often this is requested by Original Equipment Manufacturers (OEMs) and quoted for by Contract Electronics Manufacturers (CEMs) as a percentage - but what does this figure really mean? Statistics as we know can be presented in a variety of ways; it’s important therefore that any test coverage reports you receive back from your assembly partner are clear and help you to deliver the strategy you first set out to achieve.
The purpose of this blog post is to highlight some of the challenges surrounding test coverage percentage reporting, and the reasoning behind why, at best, they should only be treated as a rough guide.
Playing the percentage game
Let’s consider for a moment a humble resistor, fitted to your PCB assembly (PCBA), tested on an automated test equipment (ATE) system. For the purposes of this blog post I have assumed you have designed in a 100R component. The CEM will be able to measure the resistor on their ATE machines as 100R so that’s 100% covered, right?
Well… what if you specified a 1% resistor, but the test can only measure within a tolerance of 5%, and a 2% resistor has accidentally been fitted? What if your design requires a 5W resistor and a 1W component has been supplied by mistake? What if you want a metal film resistor but a carbon film has been procured instead? I think you can see where I’m going.
Then, there are techniques like boundary scan and vectorless tests, which essentially check that the pins of an integrated circuit are soldered down on your PCB. Even if the component is faulty, or worse still counterfeit, providing the pins are all soldered down it will pass test. So what percent coverage do you have now and what will your CEM provider be reporting back to you?
How about components in parallel, like decoupling capacitors? These are usually measured ‘in bulk’, but very often quite a few parts could be missing and the total measurement could still be within tolerance. Will your Electronics Manufacturing Services (EMS) provider include or exclude these as being ‘covered’ – and will these affect the overall percentage? Does it count if there’s optical inspection even though the device might actually be soldered incorrectly?
One of the first things a good test solution will do is check for short circuits. Some will check every electrical net against every other electrical net which is great. Flying probe machines for example, usually only check those networks that are at some point within a set distance from another, so more likely to have a short. Even if you could get your head around a coverage percentage for short circuits, based on probability and included/excluded nets, how would that figure equate to a percentage in an overall coverage report?
Very quickly a 100% test coverage statistic becomes ambiguous and worse still, maybe driving the wrong behaviour within both yours and your CEM partner’s organisation. So what else should you take into consideration to guarantee that the test strategy you adopt delivers the quality results you are looking for?
Focusing on the smaller numbers
One of the most fundamental measures when developing a test strategy is the availability of access to the electrical nets on the board. After all, if you can’t get to a component, you can’t electrically check it! So, as a rule of thumb, if you can access say 90% or more of the networks on the PCB, then you will get a comprehensive overall test coverage.
In addition, you might want to think about the critical components within your PCBA design. Of course, every component is there for a purpose, but some will be more important to the safe and successful running of your circuit than others. You may want to look at these items in more detail and consider specific testing of these key components rather than focussing on an overall report. There are software tools available, such as those offered by Aster Technologies, which can produce very comprehensive test coverage reports on a completed ATE programme. Whilst the percentages can at first glance look rather disappointing, sometimes you have to ignore the headlines and focus on the detail.
Bear in mind also that a good production test should be looking for manufacturing defects - if it’s checking the design works as well then you’ve got other problems – so emphasis should be on the most common faults such as opens, shorts, missing parts or wrong orientation.
When embarking on any test strategy with an EMS provider a lot will come down to the skill and diligence of the test development engineers. It’s therefore really important when selecting an EMS partner to critically assess the capabilities they have to offer in this area. Those who plug a board in, hit the self-learn button and then take what comes out as a given are probably not getting the best coverage possible for your products; an extra bit of thought and effort can make all the difference in delivering the test strategy you set out to achieve.
Image by Ars Electronica
