In the upcoming weblog postings I want to work out the differences between unique and non-unique indexes in SQL Server. I assume that you already know the concepts about clustered- and non clustered indexes and how they are used in SQL Server.

In the past I’ve done a lot of trainings and consulting regarding SQL Server performance tuning and it seems that some people doesn’t know the differences and implications between unique and non-unique indexes. And as you will see in the upcoming postings there are really big differences how SQL Server stores those two variants that impact the size and the efficiency of your indexes.

Let’s start today with unique and non unique non clustered indexes on a table without a clustered index, a so-called heap table in SQL Server. The following listing shows how to create our test table and populate it with 80.000 records. Each record needs 400 bytes, therefore SQL Server can put 20 records on each data page. This means that our heap table contains 4.000 data pages and 1 IAM page.

-- Create a table with 393 length + 7 bytes overhead = 400 bytes 
-- Therefore 20 records can be stored on one page (8.096 / 400) = 20,24 
CREATE TABLE CustomersHeap 
( 
    CustomerID INT NOT NULL, 
    CustomerName CHAR(100) NOT NULL, 
    CustomerAddress CHAR(100) NOT NULL, 
    Comments CHAR(189) NOT NULL 
) 
GO

-- Insert 80.000 records 
DECLARE @i INT = 1 
WHILE (@i <= 80000) 
BEGIN 
    INSERT INTO CustomersHeap VALUES 
    ( 
        @i, 
        ‘CustomerName’ + CAST(@i AS CHAR), 
        ‘CustomerAddress’ + CAST(@i AS CHAR), 
        ‘Comments’ + CAST(@i AS CHAR) 
    )     
    SET @i += 1 
END 
GO

-- Retrieve physical information about the heap table 
SELECT * FROM sys.dm_db_index_physical_stats 
( 
    DB_ID(‘NonClusteredIndexStructureHeap’), 
    OBJECT_ID(‘CustomersHeap’), 
    NULL, 
    NULL, 
    ‘DETAILED’ 
) 
GO

After the creation of the heap table and the data loading, you can now define a unique and non-unique non-clustered index on the column CustomerID of our heap table. We will define both indexes on the same column so that we can analyze the differences between unique- and non-unique non-clustered indexes.

-- Create a unique non clustered index 
CREATE UNIQUE NONCLUSTERED INDEX IDX_UniqueNCI_CustomerID 
ON CustomersHeap(CustomerID) 
GO  

-- Create a non-unique non clustered index 
CREATE NONCLUSTERED INDEX IDX_NonUniqueNCI_CustomerID 
ON CustomersHeap(CustomerID) 
GO 

If you want to define a unique non-clustered index on a column that doesn’t contain unique data, you will get back an error message from SQL Server. Important to know is that SQL Server creates a non-unique non-clustered index if you don’t specify the UNIQUE property when creating a non-clustered index. So by default you will always get a non-unique non-clustered index!

After the creation of both indexes you can analyze their size, their index depth, their size etc. with the DMV sys.dm_db_index_physical_stats. You can to pass in as the 3rd parameter the index-id. The IDs of all non-clustered indexes starts at 2, therefore the first non-clustered index gets the ID 2 and the second one the ID 3.

-- Retrieve physical information about the unique non-clustered index 
SELECT * FROM sys.dm_db_index_physical_stats 
( 
    DB_ID(‘NonClusteredIndexStructureHeap’), 
    OBJECT_ID(‘CustomersHeap’), 
    2, 
    NULL, 
    ‘DETAILED’ 
) 
GO

-- Retrieve physical information about the non-unique non-clustered index 
SELECT * FROM sys.dm_db_index_physical_stats 
( 
    DB_ID(‘NonClusteredIndexStructureHeap’), 
    OBJECT_ID(‘CustomersHeap’), 
    3, 
    NULL, 
    ‘DETAILED’ 
) 
GO

As you can see from both outputs, the index root page of the unique non-clustered index is occupied of around 24%, where the index root page of the non-unique non-clustered index is occupied of around 39%, so there must be a difference in the storage format of unique/non-unique non-clustered indexes on a heap table! In the next step we create a simple helper table that stores the output of the DBCC IND command. The structure of this helper table is directly taken from the excellent book SQL Server 2008 Internals.

-- Create a helper table 
CREATE TABLE sp_table_pages 
( 
   PageFID TINYINT, 
   PagePID INT, 
   IAMFID TINYINT, 
   IAMPID INT, 
   ObjectID INT, 
   IndexID TINYINT, 
   PartitionNumber TINYINT, 
   PartitionID BIGINT, 
   iam_chain_type VARCHAR(30), 
   PageType TINYINT, 
   IndexLevel TINYINT, 
   NextPageFID TINYINT, 
   NextPagePID INT, 
   PrevPageFID TINYINT, 
   PrevPagePID INT, 
   PRIMARY KEY (PageFID, PagePID) 
) 
GO

After the creation of this helper table we can dump out all pages that are belonging to our non-clustered indexes to this helper table with the following two calls to DBCC INC in combination with the INSERT INTO statement:

-- Write everything in a table for further analysis 
INSERT INTO sp_table_pages 
EXEC(‘DBCC IND(NonClusteredIndexStructureHeap, CustomersHeap, 2)’) 
GO

-- Write everything in a table for further analysis 
INSERT INTO sp_table_pages 
EXEC(‘DBCC IND(NonClusteredIndexStructureHeap, CustomersHeap, 3)’) 
GO

Now we can start analyzing our non-clustered indexes by using the undocumented DBCC PAGE command. You can find more information about this great command on Paul Randal’s weblog. To get some information back from DBCC PAGE you have to enable the flag 3604 of DBCC:

DBCC TRACEON(3604) 
GO

Let’s dump out the index root page of our unique non-clustered index by the following command:

DBCC PAGE(NonClusteredIndexStructureHeap, 1, 4192, 3) 
GO

This will result in the following result in SQL Server Management Studio:

As you can see from this screenshot SQL Server stores the child page of the B-tree where the minimum key of the non-clustered index is located. The child page 4161 contains for example the record with the minimum key of 540 up to the maximum key of 1078. When you dump out the index root page with the dump option 1 you get the byte by byte representation of all index records on the index root page:

DBCC PAGE(NonClusteredIndexStructureHeap, 1, 4192, 1) 
GO

SQL Server needs here 11 bytes for storing an index row. These 11 bytes are storing the following information:

• 1 byte: Status Bits
• 4 bytes: Customer ID, like 540
• 4 bytes: child PageID, like 4161
• 2 bytes: FileID, like 1

As you can see it’s up to the length of the non-clustered key how long an index row is. This also means that SQL Server is able to store more index rows on an index page if you choose a smaller non-clustered key. If you choose for example a CHAR(100) as a non-clustered index key, then SQL Server needs more index pages for your non-clustered index, which is not so efficient as using a smaller index key. The T-SQL script enclosed to this posting shows you how you can decode those bytes from the hexadecimal representation.

Finally you can dump out the child page 4161, which is located on the leaf-level of the non-clustered index.

DBCC PAGE(NonClusteredIndexStructureHeap, 1, 4161, 3) 
GO

As you can see from the figure, SQL Server stores for each index key on which data page and on which slot the corresponding record is located. Because we have not defined a clustered index on our table, SQL Server uses here the RID (Row Identifier) to point to the correct record on the data page. Index pages on the leaf-level on a heap table are different from leaf-level index pages defined on a clustered table (a table that contains a clustered index).When you dump out the leaf-level index page of the non-clustered index you can see that SQL Server needs 13 bytes per index row:

• 1 byte: Status Bits
• 4 bytes: CustomerID, like 540
• 4 bytes: PageID, like 178,
• 2 bytes: FileID, like 1
• 2 bytes: Slot number, like 19

Finally with this information in your hand, it is very easy to locate the correct record on the data page, because you know the PageID, FileID, and also the slot number where the record on the data page is located. Easy, isn’t it?

Let’s move on now to non-unique non-clustered indexes. Earlier we have already created such an index, which gets the index-id of 3 from SQL Server, because it’s the second non-clustered index we have defined. In my case the index root page of the non-unique non-clustered index is located on page 4264, therefore I dump it out with the following command:

DBCC PAGE(NonClusteredIndexStructureHeap, 1, 4264, 3) 
GO

But wait! Now the result from DBCC PAGE on the root index page on a non-unique non-clustered index is different! As you can see SQL Server returns here an additional column named “HEAP RID (key)“. The value in this column is used to make your non-unique non-clustered index unique. The HEAP RID column uses 8 additional bytes in your index row, which encodes the following information that are granted to be unique on a heap table:

• 4 bytes: PageID, like 178
• 2 bytes: FileID, like 1
• 2 bytes: Slot number, like 19

The overead of a non-unique non-clustered index on a heap table costs you 8 additional bytes per index row – on all index levels, expect the leaf-level, because SQL Server stores here always the HEAP RID as you have seen previously! So please keep this 8 bytes of additional index record overhead in mind, when you create non-clustered indexed that are NOT unique! And as I have said earlier, they are NOT unique by default!!!

In this example your non-unique non-clustered index is about 2 times bigger than the unique non-clustered index, because the unique index needs 11 bytes and the non-unique index needs 19 bytes (overhead of 8 bytes). When you look back to the output of the DMV sys.dm_db_index_physical_stats you can see that the index root page of the unique non-clustered index has a page space usage of around 24% where the index root page of the non-unique non-clustered index has a page space usage of around 39%. This will make a big difference on large non-clustered indexes!

So if you are just defining non-clustered indexes with

CREATE NONCLUSTERED INDEX ...

without thinking about the uniqueness of your data, you are wasting a lot of storage in your non-clustered indexes which also impacts the performance of your non-clustered indexes and their ongoing maintenance.
You can download the T-SQL script for this posting here.

In the next installment of this series we will have a look into the differences of unique clustered indexes and unique/non unique non-clustered indexes. Stay tuned :-)

-Klaus