CEC's Test Monitoring System and Quality Improvements

Online monitoring adopted for all CEC reference test data

The online system, similar to that used for monitoring ACEA-based engine test data, has been further developed for use with all other CEC tests following a successful initial implementation.

Introduction of the new system, which makes it easy to capture and analyse real-time reference test data, is now complete and is key to maintaining test quality.

The web-based system enables reference test results to be recorded on a continuous basis rather than the traditional manual ‘round robin’ approach. This means data is more up-to-date, is stored in one place, and can be accessed by all CEC Working Group (WG) members at any time.

The system’s user-friendliness and flexibility allows members to display, analyse and compare reference test results in graphic format across all key test parameters, making it a powerful tool for monitoring test quality and spotting data trends.

A similar system for collating data from ACEA-based CEC engine tests was successfully introduced in 1996 and has been refined and developed over the last 10 years. Five years ago the CEC Management Board set up a small team to evaluate whether this type of reference test data collection and monitoring could be adapted for the remaining 22 tests in its portfolio – including bench tests, transmission tests,
rig tests and engine fuel tests.

“Historically, this reference test data has been collected manually by working group chairman and Secretaries. It was stored in individual spreadsheets and was really only available for review at six-monthly group meetings,” explains Derek Mackney from Lubrizol, who was part of the three-man team.

“With personnel changing regularly it wasn’t easy to maintain and manage the data, and make it accessible.”

Access to real-time data

With the online system, reference test results can be uploaded and viewed at any time and from anywhere in the world where there is Internet access.

“It captures test results in real-time so they are available immediately, and the analysis tools and graphic displays make it easier to spot trends, false data or potential quality issues with a particular test,” says Derek.

“Before, it could have taken up to two years for round robin tests to be completed and the data collated – a quality issue could have arisen during that time but it might not have been discovered until much later.”

The initial system has now been superseded by an updated user-friendly system which features very powerful WebSTATISTICA charting software. The new system, www.cec-tma.net, was developed in partnership with Project Development Consultants Ltd (PDC), who support CEC’s main website and extranet.

“Our priority was to develop a system that was low-cost, user-friendly and which once
up and running could be self-maintained,” explains Chris Gray, chairman of CEC’s
Statistical Development Committee and second member of the team.

“With fewer and fewer companies employing statisticians it had to be intuitive and not
require extensive intervention from a database management company.”

Uploading data

Data is uploaded to the system via a simple Excel-type spreadsheet template, which is designed specifically for each test in conjunction with the working group. Once test results are entered into a spreadsheet and validated, an upload file is created which is then uploaded to the database website.

The system extracts the data from the upload file and registers it in the database. It provides a warning if it detects data that has not been entered in the correct format, or which is outside the normal parameters for that particular test.

“This ensures the integrity of the data being entered into the system,” says Chris. “It will also spot duplicate data if people try to enter the same data twice by mistake, which has been a common issue in the past.”

Once the test results have been uploaded, the database is created instantly and is immediately accessible to all Working Group members via the website.

Despite the system’s simplicity, it has some powerful analysis capabilities. “It’s a very flexible system which allows you to chart results graphically using a variety of different parameters including time, laboratory or reference sample,” explains Chris. “You can analyse as much or as little data as you need.”

Dynamic reports

While the system was similar to the one used to monitor ACEA-based engine tests, it pioneered an important development. “The original ACEA system generated static reports on a monthly basis but this system offers the ability to access data at any time and produce flexible, dynamic reports.

“Being able to instantly see how results have changed over time will help to spot changes in severity and identify the cause of those changes much more quickly and easily. This makes it a very effective quality monitoring tool,” says Chris.

The new system has another important quality assurance role, by providing the CEC Management Board with accurate statistical evidence that tests are being completed and results captured. “We can produce a monthly administrative report which gives an up to date picture of which tests have been run, by which laboratories, and whether the data entered was acceptable,” explains Derek Mackney.

“The database provides an accurate historical record of test results, changes in test procedures or in the quality of results. This is an important mechanism for improving both our own internal quality control and also the wider reputation of CEC test procedures.”

System Testing

The new online monitoring system was trialled in early 2008 with various test types, including CEC SG-L-039 (RE1-4 Elastomer test), CEC SG-L-014 (KO), CEC F-06- 96 (HFRR) and CEC L-40-93 (Noack Volatility test),

The tests were used to refine the process, develop appropriate documentation and improve the usability of the data-entry process.   “Once the system had been applied to the tests in the pilot, we were confident the system could be adopted for all the remaining tests,” says Chris.

“The aim is to have all CEC test procedures running on the new system by early 2009,” says Lyn Dearling from the CEC Secretariat, who are working with PDC to manage its introduction to the remaining CEC tests.

“As our experience of the system grows the migration process is becoming faster and smoother and now it is possible to complete the transition within a month or so.

“The adoption of this system will put our test monitoring up there with the best quality systems in the world. It is a powerful tool for supporting both the day-to-day activities of Working Groups and for maintaining the excellent standards that CEC tests have come to represent.”

How to find the data

Data for tests migrated to the new monitoring system can be located by logging on to www.cec-tma.net.

To obtain your user name and password, please send a request message to cec- tma@pdcnet.org.uk. Alternatively, please call Paul Bromby or James Harris on +44(0)1455 290222.