• HUN

Our solution developed by experts at Catura Consultancy PLC and ProofIT Ltd offers technological quality and reliability advantages and in most cases reduces costs. The product called ACE is a service package developed on the basis of experience at engineering and testing complex IT systems and on theories of leading international testing solutions. Its key elements are an extensive test automatization and an integrated development coupled with our specific methodology. The service package offers a quality control procedure concerning application development and operation including testing CRs and version updates. ACE works with multilayered applications in most of the standard technology environments.

 

ACE - Automated Conformance Evaluation

Automatic testing device and methodology

Testing

Testing

Quality testing in appropriate depth is an increasingly urgent need at complex IT systems at any phase of the applications’ life cycle:

Specifications are never perfectly precise and no application can entirely meet the prepared specifications. Quality control tools used at development cannot guarantee business adequacy especially when business expectations tend to change faster than the applied technology. Operating any applications will inevitably present resultant failures by numerous factors and side effects, which cannot be calculated or modelled in advance. Modifying a function or implementing a change request in an operating application should require not only testing the elements concerned but also the unchanged components.

Difficulties and Compromises

  • The more complex the processes are managed by software the more overwhelming the risks grow concerning the whole business solution. Tests usually only approve a single aspect like performance, integration or user acceptance.

  • Traditional testing costs are high. In addition, lost time during the project regularly causes time pressure. Stakeholders often compromise on the scope and quality of testing processes to save project budget or prevent project from delays. This compromise increases the risk of the “go live” decision. Unfortunately, the later failures occur the higher tracking and repairing costs rise and usually at an exponential rate.

  • Quality and business adequacy should not be separated but development phases are often cut short by business expectations. Required tests are not planned properly in advance. User acceptance is not precisely described so business adequacy cannot be measured.

  • A large number of hidden errors are left in software productions because of inadequate testing processes, regularly skipped regression tests and not checking expected rejection cases.

  • Testers are human beings and they make mistakes. Their mistakes can actually hide deeper errors in the tested system.

Aims to increase the importance of testing in accordance with business efficiency

  • Improved quality
    Testing should definitely assess technological and business adequacy.
    Testing must not change or directly impact the application.
    Testing should be repeatable, determinate and the results should be provable.

  • Running considerably more tests and covering a larger scope
    Expected rejection cases should be tested as well.
    Regression tests should be run repeatedly with no extra effort.
    Preparing test cases should not require IT expertise.

  • Reducing time and costs invested in testing
    ACE should be able to perform all operations that prepared testers can do.
    Test records and documentation should be provided automatically.
    Test cases should be generated automatically.

 

Infrastructure and Methodology

ACE infrastructure

ACE is based on Catura’s own experience in developing and operating complex IT systems and on knowledge of leading test methodologies in the IT world. There is an extensive test automatization in the centre with an integrated development built around it including elements developed by Catura. ACE exceeds previous testing products in the market but it can integrate them as testing supplements.

Automated testing infrastructure scheme:

ACE infrastructutre

Automatically generated test cases

ACE can automatically generate test cases referring to the specification of the business logic. It also inspires stakeholders to accurately define the functions and features to be developed.

All the test variables - including expected rejection cases – can add up to an enormous number. For instance, only 9 fields to fill in a window can result more than 73 million test variables. This number of tests cannot be fulfilled in practice. According to recently published studies, failures caused by 4 interacting variables (4-wise combination) statistically add up to nearly 100% of all faults even in applications of great complexity. By taking this into consideration, the number of required test cases can dramatically be reduced by using appropriate test combinations.. ACE uses this approach to reduce the 73 million cases mentioned in the example to 488 cases to test.

Methodology

ACE is appropriate for Black Box Testing when we do not know the specification and the detailed structure of the application to be tested. Test cases can be prepared based both on rules and on patterns. Regression tests can be repeated anytime in both cases and a permutation of tested values can be applied. In case of pattern based Black Box testing ACE uses a grid covering the forms of the application. Patterns are recorded using the grid fields and test cases are generated automatically.

ACE can use White Box Testing when software specifications, the structure and business expectations are all known. Besides repeatable regression tests and automatically generated test cases, testing requirements in this case should be built in the specifications. It means user acceptance will require that all test cases have had the expected results. This way ACE works as a quality control system: business functions and acceptance requirements are defined accurately and the communication between customer and provider should be refined. Business adequacy becomes measurable.

Resources invested in ACE testing

Planning and preparing test cases in ACE takes more energy and longer time than in the traditional way. Test cases have to be coded for automatic running. Then test jobs can be run at anytime and any number of times with hardly any extra costs. Test jobs will not have the human element of making mistakes and human resources can be applied to higher quality tasks.

Traditional testing Traditional testing

ACE Automatic testing ACE Automatic testing

ACE Case study

Investing 400 - 500 man months developed one of our applications called HBO. The scope is a database of 2.5 TB, 4.5 million code lines (according to testing statistics every thousand lines contain one error), more than 700 business functions, 250.000 on-line transactions daily.

Approximately 2000 test cases were prepared for 100 business functions by White Box methodology. It took 2 man months without the automatic test case generator.

Our most effective automated test job of these test cases runs for 30 minutes and accomplishes what 1 man week of manual testing can do. Other test jobs run only 10 times faster than manual testing. All test jobs can be run anytime and any number of times. Test definitions are based on business and workflow oriented approach. Valuable reports support responsible decision-making.

 

Business Advantages

Advantages for the business management

  • Accurately defined business requirements are met faster and more precisely.
  • Business adequacy becomes measurable.
  • Development and CR cycles take shorter time mitigating time pressure risks.
  • Quicker and more effective testing requires fewer resources at fewer costs.
  • Testing guarantees quality and business adequacy.
  • Self-learning capability saves time at defining and redefining business test cases.

Advantages on the operation side

  • Precisely defined user acceptance and business adequacy requirements, simpler implementations.
  • Automatic test result documentation, certain events can trigger preset alarms or actions.
  • Prepared regression tests can be run anytime and they even can be built in later CRs.
  • More tests with larger scope yield enchanced operational safety, less system failures.
  • Better defined responsibilities during development and operation.

Advantages for the developer

  • Accurately defined user acceptance and business adequacy requirements make development and acceptance more effective and the paying process simple without the usual debates and delayes.
  • More comprehensive testing means better quality and higher customer satisfaction.
  • Human resources and time invested in testing can be reduced, which allows to increase profit or to lower prices ahead of the competition.