VSLive! Speaker Interview

David Campbell, product unit manager, SQL Server Engine and Tom Rizzo Group Product Manager in the SQL Server Product Unit on the Upcoming release of Yukon.


In his keynote address at VSLive! San Francisco , “Database of the Future: A Preview of Yukon and Other Technical Advancements” David Campbell, product unit manager, SQL Server Engine, will unveil details about some of what you can expect in the upcoming release of Yukon. In a discussion with Visual Studio Magazine editor in chief Patrick Meader, Dave and Tom Rizzo, Group Product Manager in the SQL Server Product Unit, discussed Microsoft’s unified data strategy, SQL’s new integration with Visual Studio .NET, improvements to T-SQL, and more.

Meader: You’ll be speaking on the future of SQL Server at VSLive! this February. What will those who can’t attend this session miss learning about if they’re not there?

Campbell: Part of my role is to figure out what’s next with the product, and over the last year or so, I’ve gotten a handle on where databases are headed. I see a couple of big truths, and one of them is the fact that we have more and more stuff out in the real world that has some schema and structure around it, and XML is a big part of that. Databases are also becoming active in response to other things. So, you can imagine a world today where people interact with a database by putting and getting data from it, but I think we’re coming into a world where the database itself is an active part of the process. The database is an endpoint for things that are coming at it from the world that don’t go through human hands or other processes. So, you can imagine purchase orders coming in and being “endpointed” in the database, and what do you need to do that? You need to be able to handle the content by interpreting it. You’ll be able to “endpoint” it, which involves Web services and programmability within the database itself. Awareness is also a big issue—the database itself understands the content and knows that it’s not just a row in the database. It has a “personality.” For example, you’d have code within the database that understands what’s the next step in processing a purchase order as it comes into the database. So, I’ve come up with the three “A”s: autonomy, active, and awareness.

Meader: What is Microsoft’s unified data strategy, and more importantly, how will it benefit developers?

Campbell: It’s one of the things we’ve been looking at for eight years. We’d talked about unified data back with OLE DB and being able to get at all different types of data, from different sources. We invested a fair bit in that technology in the late 1990s, and I think now we’re moving towards recognizing structure and different content, and certainly XML is one of the drivers. Being able to do interesting things with XML—both in the database and in the mid-tier, passing XML payloads back and forth, and so on—is really important. We took a look at how people are investing in XML databases directly—storing everything in that format—and as we talked to people building real apps, we found it’s actually a continuum where there are applications that are, by and large, relational applications. We have decades of experience on how to build them. We have other applications that are more semi-structured in which an XML store might be the right thing. But there are a lot of very interesting apps in the middle. For example, if I were to go out and build a purchase order app today, I would do it principally as a relational-database schema, but I may very well want to be able to put annotations onto orders and line items. This way I could tell the customer, “In line item 3 there are 3,000 widgets, I can get you 2,000 of them today and the next 1,000 will come in next week.” And to be able to have an XML payload that goes along with that is really key. So, you’ll be able to bridge that gap. The work we’re doing is to be able to recognize and acknowledge schemas across that whole continuum and do interesting things with the data.

Meader: I’ve heard Microsoft talk about Yukon serving as a universal data store. How important is Yukon as a universal data store—for example, being able to store server info such as Exchange in SQL Server—to Microsoft’s long-term platform strategy?

Campbell: We’ve made a big investment in SQL over the last few years. I got here in 1994 and we’d just brought in a bunch of people from around the industry and for the first wave—from 1995 through 1997—we, in essence, built the database product. Since then the company has recognized that as an asset, and launched a second wave of products that are leveraging it ,such as Commerce Server or BizTalk Server. It’s been neat for us to learn what we need to do in terms of requirements to meet the needs of other people using the database in new ways. So, it gets to how do we store all sorts of data—different types of data, in different shapes—and what sort of things we knew we need to do within a database to act upon it. So, the two dimensions would be the breadth of the data and programming against the data—the programming against the moving bits as they come into the system.

Meader: Microsoft indicates that the next version of SQL Server will be tightly integrated with the next version of Visual Studio .NET. What forms will this integration take?

Campbell: As you’re probably aware, we’ve integrated the common language runtime (CLR) within SQL Server. So, that’s the first step. One of the key strengths that we’ve had is the development environment and the development experience around Visual Studio. Developers will be able to go out and build a data-centric application and use some of the new Web services tools and endpoint them directly into the database.

Rizzo: There’s a bunch of other work that we’re doing for which the CLR was the key integration point, and that brings a ton of benefits, because developers who have experience in developing for the mid-tier, let’s say in VB or C# for their Web applications, can now write their data tier-based applications in the same languages. They can build stored procedures using the .NET common language runtime in any of the languages that we support.

Now, one key thing that people should realize is that T-SQL doesn’t go away; T-SQL actually gets enhanced in this timeframe as a development language. You’ll see additions such as structured exception handling in T-SQL, and try and catch blocks will be supported there as well. Plus, a lot of customers will move to the CLR, quite possibly, if they have the investment already in the development skills that the CLR provides.

Campbell: So, if I have a purchase order that’s described in an XML format and I can endpoint that directly into the server, then if I can write my business logic around that directly in whatever my most appropriate language is to interact with it. I think that becomes really interesting in that sense it brings that all together as an integrated package.

Meader: Let’s drill down on that a little bit. You mentioned that the runtime will be included in Yukon. I assume there will be both the stand-alone version of SQL Server or Yukon and another version that plugs directly into the Visual Studio .NET IDE.

Rizzo: We’ll have one version of SQL Server and that version of SQL Server will integrate with all versions of Visual Studio. For example, the Visual Studio .NET RTM could actually work against Yukon.

Meader: So what’s your working environment? If I’m a Visual Studio .NET developer do I fire up Visual Studio .NET and create my SQL application from there, or do I fire up SQL Server?

Rizzo: You can do either. It’s similar to the model we have today. You can use Query Analyzer inside of SQL Server. So if you’re doing a bunch of T-SQL work and you want to run a query and get back results, you probably wouldn’t develop your ASP .NET application in Query Analyzer; you’d use a professional development tool. If you’re a professional developer, Visual Studio .NET is your development environment. If you’re a DBA and you just want to create some scripts or run Query Analyzer to write some quick T-SQL to get back a result, or you want to administer your databases, then you’re going to use the SQL Server tools.

Campbell: I think there are a couple of key points to go along that. One is the end-to-end integration, and this is something we’ve been working for some time. This allows you to debug through the Visual Studio CLR code, and actually through to the T-SQL side, which is very cool in terms of being able to jump directly into a query and see the results of that query and back out into the code. The other point is if you’re doing data-centric programming. When I write a T-SQL stored procedure, it’s a data-centric procedure, and now with the integration, you can write that in your language of choice. What we’ve tried to do is make the deployment decision—whether I want to run that in the mid-tier or within the server—rather than making that a design-time thing, something you need to think about up front, to make it generally a deployment-time decision. So, you can imagine, you get benefits when you run code within the server because you can avoid some latencies—you don’t have the network round-trips and whatnot—but then you may get to a point where it makes more sense to deploy it in a mid-tier, and you can do all of that just by changing the connect strings. So, I think there’s much more freedom in terms of where you’re going to be running that. And, so, if you’re an ISV, in smaller configurations, you can certainly run everything within the server in one process and get some efficiencies there. And then as you scale it up and out, you can take that same code and run it out in mid-tier and replicate it across a number of mid-tier servers.

Meader: You’ve mentioned you’ll be able to write stored procedures using either C# or VB .NET, much as you can write stored procedures using Java when you’re using Oracle. What are the advantages and disadvantages of being able to do this?

Campbell: One of the advantages is you get all of the structure that you get within the languages, plus all the other facilities. T-SQL has a lot of facilities for querying the database itself, but it’s not very good for doing numeric computation or other sorts of data transformations within the data. So, you get all those facilities from whatever language, and you get skills transferability as well. If you know C# or you know VB very well, you can transfer those skills over. You express the queries just as you would otherwise—selecting what data, what joins you want to do, and so on—but once you get the data back, the binding is much easier and you have all of the facilities of your language of choice available to you within the server itself. So, that’s a key advantage.

The only disadvantage would be, because you get lots of flexibility, just figuring out what you want to run and how you want to factor your application, which piece will run in the mid-tier and which piece will run in the server. But I think it is an opportunity as well.

Meader: Are there any performance implications of writing the stored procedures in C# or VB .NET versus T-SQL?

Rizzo: Well, there are actually several interesting performance parameters. When you run something within the server you’re running all within the same process space. So when you do transitions in and out of that piece of code, they’re much more efficient than if they were interacting across the network. So, if you’ve done any two- or three-tier application development one of the key metrics for performance is minimizing the number of round-trips or requests to the server. The cost for that request almost goes to zero when you’re running inside the server, so you get some efficiencies there. The other thing we get in T-SQL is an interpreted language in some sense. We get the benefit of being able to run the CLR and all the JIT compiling technologies, so there are some constructs and expressions that will just run faster in the CLR than they would in equivalent form in T-SQL, and we’ve demonstrated that in-house.

Meader: In a similar vein, does this scenario create maintenance issues such as when developers are allowed to use different languages to code stored procedures? For example, assume you have an employee who writes all his stored procedures in VB .NET, but he leaves and the next person you hire is stronger in T-SQL or C#. Do companies benefit by standardizing on T-SQL instead, which is more widely used?

Campbell: This question comes up a lot. In any development environment or shop, you have a decision to make: Do you allow your developers to go out and pick their language of choice or do you have other procedures in place? I suspect that out in the real world, there are probably ten times as many C++ or C# programmers as there are T-SQL programmers, and the same thing with VB. So, if your people have a lot of skills then I think it’s just a matter of what sort of development environment and what sort of processes you want to put in place. If I were in that camp, I’d certainly want to make a decision in terms of what I want to use and standardize on that.

Meader: How important will it be for system administrators or database administrators to understand Visual Basic .NET or C#, given the integration taking place?

Campbell: In many shops, there are people who develop the queries and then there are database administrators who do the “caring for and feeding” of the server. One of the things that we’ve seen with SQL Server over the last couple of years is that because we invested heavily in making the server autonomous and self-healing and self-monitoring, we have a number of cases where there were DBAs who were turning configuration knobs five years ago who are now, in some sense, “upskilling” themselves to the point where they now help developers write queries. And, so, there’s less care and feeding, there’s less pagers going off, but I still think there’s a distinction between people who are developing and optimizing queries and the people who are developing and optimizing the rest of the code.

Meader: How significantly will the changes in Yukon affect the existing data model. For example, what kind of issues will existing SQL admins and developers face when upgrading to Yukon?

Campbell: By and large, we’ve invested heavily over the years just to make the upgrade completely compatible. If you look at the transition from SQL 6.5 to SQL 7.0, we effectively rewrote the entire engine, but our T-SQL compatibility was 100 percent. So, we try not to leave any gaps or discontinuities. We want to be able to have people adopt and take things in as it makes sense. So, an example would be the use of XML within the system, or all of your T-SQL stuff just runs exactly as it is today and then you can migrate over to using some CLR stored procedures and other Web services type stuff as it goes on. You get complete backwards compatibility. You also get a choice in how much you want to adopt and what makes sense for your environment.

Meader: Microsoft is touting improved security for SQL Server in the Yukon release. In terms of features, what exactly will make Yukon more secure?

Rizzo: There are three key things that we talk about for Yukon. First, it’s secure by default. So, we install with the services turned off—like Windows Server 2003—you have to enable all of the features that you want on the server, so there’s less surface area for attacks by hackers. Second, we’re secure by design. When we’re actually building Yukon at Microsoft, we make sure that we do threat analysis, that we design our components with malicious attacks in place in our design, and we test the actual product as we build it. Finally, we’re secure by implementation. At runtime, we do a lot of work in Yukon to make sure that you’re not shooting yourself in the foot from a security standpoint. So, for example, today we don’t recommend that you use SA as the username with no password. There was a SQL worm going around hitting all servers looking for SA, no password. In Yukon, we will still let you do SA, no password, but we will warn you multiple times that you probably do not want to do that, because it is a security risk. But for compatibility, we have to allow you to use SA, no password, just in case it breaks your apps.

And there’s another security feature we’re adding, which is row-level security. Today, we do security at the view and at the table level, and so a lot of customers asked us to implement security at the row level so you could lock a certain row based on the user, so it doesn’t even appear back in the result set, nor can they modify values in that row.

Meader: Apart from the security and the .NET integration, what else is new in Yukon that we might not all have heard about?

Rizzo: One of the key new things is our XML data types. We will have native XML support directly inside of Yukon. Today, in SQL Server 2000, we have XML support using the Web Services Toolkit, so we can expose stored procedures as Web services from the mid-tier as well as support XML bulk load and shredding of XML data into the database. With Yukon, we will have native XML support, and we will have native Web services support from the server side. So it doesn’t require a mid-tier IIS server; your server can actually publish Web services directly out of the SQL Server. The other exciting features inside of Yukon have to do with our business intelligence. We’re doing end-to-end business intelligence inside of Yukon. You’ll see advancements to our analysis services, to our MDX or multi-dimensional querying extensions that we have for T-SQL, and then you’ll see some new technology that we can’t talk about right now, but it will provide an end-to-end business-intelligence data-warehousing solution built on top of Yukon. And we are improving our enterprise abilities, and we will improve our enterprise abilities with every release. We’re focusing really heavily on scale-up performance, so we’re doing a ton of work with 64-bit, especially with SQL Server 2000 in the short term, and with Yukon in the longer term to make sure that we scale up on very large boxes so that customers can build very complex, mission-critical applications on top of the SQL Server platform. And we are making advancements in our high-availability story as well. Today, a lot of customers build log shipping to do high availability between two SQL servers so that if one fails, you can connect to the other SQL server and continue going. We are making advancements to that high-availability story with something that is called “database mirroring.” You will have a mirrored copy of your database on a separate server and your clients will fail over automatically if the original server goes down.


David Campbell is the Product Unit Manager for US-SQL Server Engine. He is a software architect responsible for the SQL Server Storage Engine. He has been with Microsoft working on SQL Server since SQL 6.0. David is currently working on Microsoft's unified data strategy and SQL Server's strategic evolution

Tom Rizzo is a Group Product Manager in the SQL Server Product Unit. Tom focuses on emerging technologies in the SQL Server group including the 64-bit version of Microsoft SQL Server 2000 and the forthcoming Yukon release of SQL Server. Before working in SQL Server, Tom worked in the Microsoft Exchange Server product group.