Now Showing! The Right Process For Verification IP Development

This blog is useful for all those who are familiar with verification development – but want to get better at it. It is useful for design engineers as it will help them evaluate (and push) their verification teams to deliver a better solution. It is useful for verification engineers also, as it will help them better their skills. If you are a beginner - please refer to our blog on Basics of Functional Verification as well.

Why is it important to have a great process for development of Verification IP?

Verification plays a major role in any chip design project. Within that, functional verification takes lion’s share of any design cycle, 70% by some estimates. Functional verification completeness is extremely critical. Following the wrong process for developing verification IP can cause you to miss verifying some features. This can cause chip failure and cost the company millions of dollars. It has been noted that close to 70% of chip re-spins are typically due to functional bugs although that percentage has come down to ~50% in recent times. Thus following a good process with lots of planning is essential for development of world class verification IP.

Don't “Methodologies” already exist?

Standard verification methodologies such as VMM, OVM and UVM mostly focus on the test bench building methodology rather than the process of building a complete Verification IP. Although there have been attempts to standardize the process of the building Verification IPs but they have been in there early stages. Hence there is need for a more complete methodology approach towards building Verification IPs.

So what’s the right process for developing a Verification IP? Is there any?

Yes there is! And we call it CVM - Comprehensive Verification Methodology. So here we are sharing with you what we think is the right process one should follow to come up with best-in-class Verification IPs. Using this process we have developed world class Verification IPs, time and time again. For more on our verification IPs, click here.

Overview of Comprehensive Verification Methodology (CVM)

The process can be broadly broken down into three stages: Planning, Implementation and Closure.

Planning:

Planning is intellectually demanding process. Planning properly not only makes objectives clear but also offsets the uncertainty that goes with any project. So what steps are involved in planning a Verification IP? We have broadly classified them into the following:

• Scan Specification: As a first step we need to have some knowledge of the “protocol” for which we are developing the Verification IP. So scan the full specification and try toget a fairly good understanding of it. Goal of this step shouldn’t be to understand the specification to the minutest details but to get an overview of the protocol as a whole. As you proceed further you should keep referring to the specs as and when needed.
• Define Plans: Once you have had a good look at the spec and understand most of it, it is time to create plans for each defined layer. Say for example in USB you can have plans for Protocol, Link and Physical layers. So depending on the need come up with plans according to the spec in question. So what are these plans?  They are of three types:
  • Test plan: It encapsulates the features that are mentioned in the spec. This plan captures the tests that need to be written in order to verify all the required features and sequences. Here are some test plans to get you started.
  • Checks Plan: Typically the spec has clearly defined values for some parameters and variables and some defined sequences. So this plan tries to capture all such cases according to the spec. 
  • Coverage Plan: This plan captures all the parameters and sequences that need to be hit as per the spec to achieve 100% functional completeness. 

• Define TB/BFM architecture: By this time you would have scanned the spec multiple times and you would have some understanding of different layers. At this stage, you need to define the Test Bench/BFM. You need to ensure that this is fool proof and can stand the test of time. Some thought should go into developing this. Wrong architectures are always dangerous! They come back and haunt you at a later stage when the Verification IP starts taking shape and it can become painful to fix it at such a stage. So think and define!

• Create TODO list and Milestones: Once you have defined the architecture and have the plans ready, you will have afair idea of what you and your team need to do for developing the verification IP in question. For effective implementation, you now need to assign ownerships and timelines. Also define a TODO list of any missed items so that it can be looked into at an appropriate time. This is an important step as it helps pace the development of the Verification IP. It also helps in checking if you missed some items/elements while coming up with either plans or TB/BFM architecture.

Implementation:

After creating the plans, architecture and milestones it’s time to press the start button and begin development of the test bench and BFM!

• Architecture review: Before you jump into the implementation stage you should always review the architecture that you defined earlier. This is done to ensure that going forward there aren’t any major surprises and your architecture stands the test of time. Core of the architecture should be strong and clean.
• Define TB/BFM interfaces: Define the virtual interfaces that will be exposed to DUT and alsoother interfaces (if there are any).
• Define Abstract Test Bench classes: Define the placeholder structures/classes for environment, agents, drivers, sequencers, scoreboard, monitors, checkers etc. This definition will depend on the test bench methodology that you are using. Here for example we have used UVM test bench methodology.
• Define Abstract BFM classes: Define key properties, placeholder members of the different classes (e.g. config, data etc)

Closure:

Once you have laid the groundwork and provided shape and framework for the verification IP, you can march towards the next stage i.e. filling the framework with required content to make it meaningful. At this stage you should update the BFM classes, create scenarios by writing tests using test bench infra and march towards coverage convergence i.e.100% coverage.

• Normal Operations: As a first step you should be able to get the normal operation working. This might require implementation of the BFM class members with required logic, writing of appropriate tests based on the test plan and implementation of checks, as needed. Normal operations might differ across protocols. Remember however, this doesn’t coverany error injection or low power related mode.
• Error Injection Operations: Once you have the normal mode working, you should target error injection scenarios. You should follow the same steps as followed for normal operation and cover all the cases as captured in the test and checks plan.
• Low Power and Miscellaneous Operations: Now that the normal and error injection operations areworking, close to two-thirds of the job is done! Next target is to verify low power features. This needs dedicated testing. We follow the same process as followed for the above two modes. At last comes verification of any other miscellaneous features that are pending from the test plan in order to get to functional completeness.
• Coverage Convergence: This is a very important step as it gauges the functional completeness of the Verification IP. This is like checking the score after writing the exam! The only difference is that you have a chance to improve the score systematically if you have a plan. Here is where the Coverage Plan comes handy as it helps in recognizing the missing parameters or sequences that were not hit or tested. To get to 100% coverage, you may have to write some extra tests to create scenarios that address any missed bins or features. Here is a blog on The Secret Sauce of Functional Coverage Plan Management. Here is another blog that helps you automate functional coverage management.
• Code Reviews: As seen from the diagram it is good to start doing code reviews at an early stage just as you start hitting the normal working mode and start filling up the infra. Code reviews are always important and they help address the following issues:
  • Is the code functionally correct and is it doing what it is supposed to do?
  • Can it be maintained and reused?
  • How clean and well defined the code is? It is aesthetically clean?

Following this approach not only helps in debugging the issues easily but also helps in abstracting the code so that people can easily relate to it.

So there we go! We strongly feel that following the above mentioned process not only makes Verification IP development easy but also increases the probability of coming out with world-class Verification IP’s and that too, on time! Using an effective verification process eases Verification IP development by giving the verification team a clear picture of the task at hand. It also improves the quality of the Verification IP by ensuring timely execution and repeatability of building successful Verification IPs again and again.

Does your design verification team follow a set process for developing Verification IPs? Any experiences you would like to share? Let us know in the comments below! 

Meanwhile, here is an experience that recently happened somewhere ...