导航:Electronic News sat down to discuss the future of verification with Scott Sandler, president and CEO of Novas Software; Mark Gogolewski, CTO of Denali; Farhad Hayat, VP of marketing for Synopsys'' verification group, and Lauro Rizzatti, general manager of Emulation and Verification Engineering USA. What follows are excerpts of that discussion.
Electronic News: How big is the verification problem these days?
Sandler: It''s bigger than it''s ever been. It''s a constant increase based upon Moore''s Law. There is no change in that. I''ve been involved in this since 1983, first as someone trying to do verification and then with simulators and formal and now debug. There are some big themes that continue. One is shifting of levels of abstraction, so we see a lot of work with ESL [electronic system level design], which I like to characterize as the most recent, favorite level of abstraction to develop commercial automation. There''s also refinement around RTL [register transfer language], and work closer to the silicon. Bigger chips are harder to get right at the detail level, so there''s more work to tape out and even after tapeout. The problem is bigger, but the process hasn''t changed fundamentally.
Gogolewski: How bad the problem is depends upon whose seat you''re sitting in. Legacy philosophies tend to get you into more trouble. Those teams that look at design from a holistic view of re-use have a much different success rate and risk profile. If you look far enough back, it was looking at the best way to design, then hand it off to get tested and then figure out how to build the software. The most successful teams today look at what are the right decisions from a software point of view—that can be both re-use and capability—and what techniques help tame the verification problem. When those considerations take priority, then figure out the best way to design. Verification is 70 percent of the battle for every new IP block. That means your priority has to be what''s the right decision to contain the verification cost and the risk.
Hayat: The complexity has grown. But when you look at the profile of what goes into verification, customers are looking at more tasks to track, more things to plan. It''s not just that you have to run more of the same. The number of things that have to be tracked across the verification path has become larger. There are a lot of opportunities for productivity beyond tools, including integration of how tools work together, as well as tracking the verification process and having a system that looks at everything at the same time. For many of our customers, it''s not whether they can get the verification done, it''s a question of how they can shrink this cycle. More sophisticated customers have put together technologies and techniques to get the job done. But how do they make it more productive? The market is starting to adopt assertions, test-bench automation, IP re-use. All of these things help to contain the problem. But the enormity of what they have to track is one of the top issues for our customers.
Rizzatti: There is another dimension to the problem. Unlike ASICs, today there are SoCs. ASIC design starts are dropping. Today there are about 2,000. There used to be about 13,000 a year. FPGAs are hitting probably 100,000. What I''m talking about here is the SoCs. There is lots of embedded software. When you hit 0.13 microns, the software effort to develop and test the embedded software exceeds the validated hardware. At 90 nanometers, it''s worse. There''s a compound issue of not only testing hardware, but testing the integration between hardware and software. That makes everything an order of magnitude more complicated.
Electronic News: If verification is 70 percent of the time of developing a chip, how much of the cost is it?
Sandler: There''s the fixed cost and the part you pay your people. If you''re talking verification specifically, it''s all people costs. It''s enormous. Out of your people costs, it''s 70 percent. If you look at the overall cost of design, it''s those people plus the masks plus your capital equipment.
Rizzatti: There''s also the missed revenues if you miss the market window. Three months late is all of the revenue for a product.
Sandler: For some of our customers, that can put them out of business. With the game companies trying to hit Christmas windows, if their verification causes them to slip they could lose that whole product cycle.
Gogolewski: We''ve seen plenty of companies trying to figure out which divisions to keep and which ones not to. It''s kind of like skydiving. If you don''t pull your chute at the right time, it''s over. When customers lay this stuff out on a spreadsheet to determine when all of this stuff is going to get done, the part that scares them most is the human effort. When you think about this, the last thing you really want is to do the human effort. It adds uncertainty into the cost. Re-use is really a key factor. So is automating things that should never be done by an engineer. Some of the things we''re seeing in ESL address that, such as stitching together SoCs automatically and managing the chip comment.
Electronic News: Let''s go back to something that came up here before. What''s changed in this process?
Sandler: Fundamentally, people are still simulating and writing test benches at some level. You can even think of software as the new test bench, because now it''s the real software running rather than you coming up with it. At the core of it, though, it''s still lots of simulation.
Hayat: What''s changed is the ad hoc approach. More customers realize they need to put methodologies up front. Where verification used to be an afterthought, now it''s front and center at the beginning of the project. Customers have realized that to manage it, they need a methodology in place. In the past, you might have been able to get away with having the design engineer do the verification. Over time, verification teams have been formed that are independent of the design teams.
Hayat: It can make it harder if you don''t do it the right way. IP needs to be coupled with verification suites. It allows you to verify your IP in the context of the system it''s plugged into. You need to have a self-contained verification that can exercise your device, both internally and when it''s plugged into a bus structure so you can verify the protocols. It''s important to have standards-based IP so you can plug and play with standard APIs and not have to re-invent the wheel every time.
Gogolewski: When we talk to our customers about a new IP product, most commonly what they want to see first is the verification plan—more than the spec and more than the RTL. You need to know how to test this. There is so much complexity and so many blocks on these chips. You have the ability to create stuff you cannot yourself come up with.
Sandler: Even though it has the promise of making things easier, IP adds complexity, which also adds new risk. If you''re going to plug something in that validates whether your design is compliant with USB, how do you know you can trust the verification IP? Over time, we will come to have trusted verification IP that is proven in the industry. But the protocols are always shifting, so if you''re building the latest, greatest thing, there is no proven verification IP.
Electronic News: If you introduce new IP, are you adding unknowns that may not show up in verification?
Hayat: That''s a function of what kinds of verification techniques you use. If you use constrained random verification, where you use test-bench automation tools to thoroughly exercise the verification IP and verify the protocols based upon what the specs say, that is critical for quality IP. Once it''s plugged into a system, you have emulated any kind of stimulus that could be coming to the IP.
Sandler: There''s nothing new here. We''ve just automated it more commercially.
Sandler: It depends on what you''re designing. We used to do complex bus protocols 20 years ago. They didn''t run as fast and the chips weren''t as big, but the problem was complex even then. Everything has shifted up. Techniques are better and software is available.
Hayat: There are still a lot of customers that have not adopted these techniques. Maybe the leading-edge customers did. But what''s different is that these techniques are moving mainstream. They''re not just reserved for the customers that have the budgets. It''s now available through System Verilog, test-bench automation tools, assertion-based verification libraries and verification IP. It''s not so much that we''ve found a new silver bullet. It''s that everything builds upon what was already there. The techniques were available many years ago, but now they''ve become commercially available.
Rizzatti: Ten years ago, 1 million cycles of verification would do the job. Today, you just scratch the surface. You need billions, and maybe even 1 trillion, depending on the size of the design. Using conventional simulation doesn''t make it. We did some testing of verification IP for PCI Express, which is a very complex block. The performance was ridiculously slow. There are new methodologies, but performance is poor. That''s why it''s 70 percent of the time. It''s performance.
Electronic News: Can you verify the entire chip at once?
Rizzatti: Certainly you do the system-level verfication, but using a simulator it''s inconceivable.
Hayat: If you don''t test the blocks and throw everything together, that''s a recipe for disaster. A bottom-up approach, where you verify at the block level and make sure the block is fully verified, it helps. As you put it together, you have to keep verifying until you get to the chip level. Many of our customers use large CPU farms, running in parallel overnight, to get the verification done. If you look the leading-edge customers, they do assertion-based verification, they have simulation farms, and when it comes to system validation they may use simulation because they want to run software or long sequences of verification.
Sandler: If the cost is in the people, you can speed it up to find a bug but it still takes a long time to find out why. That''s part of that 70 percent. You can launch on the farm or the emulator and run more random tests, but you''ve always got to debug it.
Electronic News: If the cost is in the people, does that mean you have to move the operation to areas with less-expensive labor?
Hayat: It''s a distributed task. Verification is farmed out to different groups. Sub-assemblies are verified in the U.S., Europe, India, China, where the expertise is. It is not like everything is moving to one area or another. Because of that, you need a much more structured approach. It''s no longer the guy down the hall. You need well-defined specs, well-defined starting and end points. If you don''t have that, farming it out to different locations is not an option.
Sandler: People don''t like to have things thrown over the wall to them overseas any more than they do here. If you put your verification team offshore, you don''t get great results. You need more integrated teams. They want to own projects. What''s happening is the whole process is being distributed around the world, not just verification.
Gogolewski: Verification takes a whole patchwork of methodologies, skill sets and tools, whether it''s internal or externally focused. There''s a huge education that has to happen around the industry. It''s a rare person who is an expert in all the verification techniques. Emulation has a great value, but that itself won''t get you there. You can''t just throw your design on FPGAs. One of the great strengths of a software environment is that you can do anything.
Rizzatti: Given enough time, yes.
Gogolewski: Around a block and a small sub-system, you can cause events to happen and uncover issues far faster than you can do in terms of the iterative debug and fix cycle. That is not going away, especially with something new. If you''re re-using a block, you probably can get away with barely touching it before throwing it into a broader system verification.
Electronic News: How do you speed up verification?
Hayat: Methodology. It''s very tempting to dive into verification because you see results. You get to a point where you have diminishing returns. Running more cycles doesn''t help anymore. You need a methodology to build an environment, integrate verification IP and make sure these IP blocks talk to each other. Then you can accelerate the productivity much faster. The way to do it is with a structured methodology.