具有多個文件的文件組:移動內容時的奇怪行為
我目前正在將數據庫從 SQL Server 2008 (SP4) 遷移到 SQL Server 2017 (CU3)。一個顯著的變化是所有文件組在遷移後都有兩個數據文件。為了完成這個任務,我恢復了一個備份,添加了具有兩個相同大小的文件以及相同的自動增長設置的新文件組,並使用以下語法傳輸數據:
CREATE UNIQUE CLUSTERED INDEX <PK of the table> ..... WITH (DROP_EXISTING = ON ,...) ON <new Filegroup>
不幸的是,我還必須移動一些 LOB,然後事情變得有點複雜:
- 添加分區方案和分區函式(基本同目標)
- 在新分區方案上創建聚集索引 (WITH DROP_EXISTING=ON)
- 在新文件組上創建聚集索引 (WITH DROP_EXISTING=ON)
- DROP 分區方案和功能
Kimberly Tripp在這裡描述了這種技術,並且可以追溯到 Brad Hoff。
感謝您的關注,現在回答我的問題:
通過像這樣重建索引,文件組中需要多少可用空間?
我給你舉個例子:
- 我有一個大小為 220GB 的 LOB 表(根據 sys.allocation_units 中的總頁數除以 128 除以 1024)。
- 將我的新(空)文件組(僅託管該表)中的兩個文件預調整為 220/2 = 110 GB,每個文件增長 = 0。
- 嘗試使用上述技術傳輸表,但收到錯誤消息
無法為數據庫 ‘abc’ 中的對象 ‘xyz’.‘xyz_pk’ 分配空間,因為 ‘def’ 文件組已滿。通過刪除不需要的文件、刪除文件組中的對象、向文件組添加其他文件或為文件組中的現有文件設置自動增長來創建磁碟空間。
- 將文件大小增加到每個 112 GB…再次嘗試並得到相同的錯誤消息
- 最後打開文件的自動增長,過程成功完成
但是**,每個文件的大小為 220 GB,每個文件中留有50% 的可用空間**。
這是建議的診斷查詢 的輸出文件大小加起來為 227,22 GB。
到目前為止,我不知道有什麼其他補救措施
DBCC SHRINKFILE
可以擺脫荒謬的大量可用空間。然而,這並不是我特別自豪的事情……這需要很長時間,可能會留下腐敗等等。你能幫我理解為什麼 SQL Server 分配了這麼多可用空間,然後快樂地按比例填充我的兩個文件嗎?
之後我會嘗試準備一個展示……對不起,但我現在時間不多,也許你們中的一些專家已經知道這樣做的原因。
在此先感謝您的幫助
馬丁
首先感謝你們幫助我跟上你們的評論。
我現在已經完成了一個範例,並且對正在發生的事情有了更好的理解。
將 LOB 數據(例如 VARCHAR(MAX)、XML 等)移動到另一個文件組時會出現問題。當您在另一個文件組上重建聚集索引時,LOB 數據將停留在它原來的位置(由
TEXTIMAGE ON
CREATE TABLE 語句中的命令設置)。移動 LOB 數據的一種經典方法是在新文件組中創建具有相同結構的第二個表,複製數據,刪除舊表並重命名新表。然而,這會帶來各種可能的問題,例如失去數據、無效數據(由於缺少檢查約束)和錯誤處理非常困難。我過去曾為一張桌子這樣做過,但恕我直言,它不能很好地擴展,考慮到不得不轉移 100 張桌子的噩夢,你得到了要修復的桌子 15、33、88 和 99 的錯誤。
因此,我使用了一個眾所周知的關於分區的技巧:正如Kimberly Tripp所述,當您在新文件組上進行分區時,LOB-Data 確實會移動到新文件組。因為我不打算從長遠來看使用分區,而只是作為移動 LOB 的助手,所以分區方案非常乏味(將所有分區放入一個文件組):我什至不在乎數據在哪個分區上因為我只是想讓他們感動。實際上,這種技術和實現並不是我自己發明的……我使用了Mark White的強大腳本。我的錯誤是沒有完全理解這個腳本的作用和含義是什麼……我現在有:
對於 LOB-Data,有必要重建(或重新創建)表(主要是聚集索引)兩次:第一次在其上放置分區,第二次刪除分區。無論你是否使用
SORT_IN_TEMPDB=ON
這都會導致必須兩次提供原始數據的空間:如果你的原始表有 100MB,則需要提供 200MB 才能使操作成功。一開始我很困惑,最後我的新數據文件在操作完成後有很多可用空間。現在我接受了我不能欺騙避開可用空間。但是,我可以避免事後縮小文件的必要性。因此,我的解決方案是在臨時文件組上進行第一次重建,在目標文件組上進行第二次重建(刪除分區)。之後可以刪除臨時文件組(如果希望我不再遇到錯誤消息“無法刪除文件組”(請在此處查看我的問題)。
感謝您的閱讀和幫助
馬丁
這是我的問題的重現腳本,其中包括我為此提出的解決方案:
/*============================================================================ Adapted the following file published by sqlskills to demonstrate filegrowth after partitioning for StackOverflow Question. Martin Guth, 02.02.2018 File: MovingLOBData.sql Summary: Because 2012 supports online index rebuilds - even with LOB. You might think this means you can move LOB data around (one of the VERY cool things you can do with IN_ROW data to actually move it). However, the behavior of LOB data is NOT necessarily intuitive. This script will show you how/why/what! SQL Server Versions: SQL Server 2012 ------------------------------------------------------------------------------ Written by SQLskills.com (c) SQLskills.com. All rights reserved. For more scripts and sample code, check out http://www.SQLskills.com You may alter this code for your own *non-commercial* purposes. You may republish altered code as long as you include this copyright and give due credit, but you must obtain prior permission before blogging this code. THIS CODE AND INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. ============================================================================*/ SET NOCOUNT ON go USE master go CREATE DATABASE [TestLOB] CONTAINMENT = NONE ON PRIMARY ( NAME = N'TestLOBPrimary' , FILENAME = N'U:\DB_DATA\TestLOBPrimary.mdf' , SIZE = 100MB , FILEGROWTH = 1024KB ), FILEGROUP [FG1] ( NAME = N'FG1File1' , FILENAME = N'U:\DB_DATA\FG1File1.ndf' , SIZE = 40MB , FILEGROWTH = 20480KB ), ( NAME = N'FG1File2' , FILENAME = N'U:\DB_DATA\FG1File2.ndf' , SIZE = 40MB , FILEGROWTH = 20480KB ), FILEGROUP [FG2] ( NAME = N'FG2File1' , FILENAME = N'U:\DB_DATA\FG2File1.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), ( NAME = N'FG2File2' , FILENAME = N'U:\DB_DATA\FG2File2.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), FILEGROUP [tempLOB] ( NAME = N'tempLOB1' , FILENAME = N'U:\DB_DATA\templob1.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), ( NAME = N'tempLOB2' , FILENAME = N'U:\DB_DATA\templob2.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ) LOG ON ( NAME = N'TestLOBLog' , FILENAME = N'U:\DB_DATA\TestLOBLog.ldf' , SIZE = 10MB , FILEGROWTH = 10MB) GO USE TestLOB go ALTER DATABASE TestLOB MODIFY FILEGROUP FG1 DEFAULT go --DROP TABLE TestLobTable; CREATE TABLE dbo.TestLobTable ( c1 int identity, c2 char(8000) default 'this is a test', c3 varchar(max) NULL ) -- will be created on FG1 go INSERT INTO dbo.TestLobTable ( c2, c3 ) VALUES ( 'this is a test', REPLICATE (convert(varchar(max), 'ABC'), 8000) ) go 1000 CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) go sp_blitzIndex @databaseName = 'TestLOB', @schemaName = 'dbo', @tableName = 'TestLobTable' go -- size is roughly 40 MB: 1,000 rows; 39.2MB; 31.3MB LOB SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 2.875000 FG1File1 5120 40.000000 19.687500 FG1File2 5120 40.000000 19.687500 FG2File1 2560 20.000000 0.062500 FG2File2 2560 20.000000 0.062500 --> 2*19,687 MB are occupied in Filegroup 1 ---> approx 40 MB in total in Filegroup 1 */ /* moving Lob data using partitioning trick */ CREATE PARTITION FUNCTION PF_TestLobTable (int) AS RANGE RIGHT FOR VALUES (0) go CREATE PARTITION SCHEME PS_TestLobTable AS PARTITION PF_TestLobTable TO ( fg2, fg2 ) go CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON, SORT_IN_TEMPDB= OFF) ON PS_TestLobTable (c1) go SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 2688 21.000000 19.750000 FG2File2 2688 21.000000 19.687500 --> now Filegroup 2 has roughly 40 MB data...interestingly the create index would fail if having 2*20MB capacity available but would pass at 2*21MB */ -- try to recreate the index again to get rid of partitioning CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON, SORT_IN_TEMPDB= OFF) ON [FG2] go /* error message 1105 Could not allocate space for object 'dbo.TestLobTable'.'TestLobTableCL' in database 'TestLOB' because the 'FG2' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup. --> makes sense because no free space available in filegroup with partitioned clustered index already present */ ALTER DATABASE TestLOB MODIFY FILE (NAME = N'FG2File1', SIZE=41MB); ALTER DATABASE TestLOB MODIFY FILE (NAME = N'FG2File2', SIZE=41MB); -- rebuild again without sort in tempdb CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON, SORT_IN_TEMPDB= OFF) ON [FG2] go SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 5248 41.000000 19.750000 FG2File2 5248 41.000000 19.625000 --> now the files of FG2 have 50% free space left */ -- try to shrink with truncateonly DBCC SHRINKFILE('FG2File1', TRUNCATEONLY); DBCC SHRINKFILE('FG2File2', TRUNCATEONLY); SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 5048 39.437500 19.750000 FG2File2 5040 39.375000 19.625000 --> no significant effect...still almost 50% free space */ -- recreate the table DROP PARTITION SCHEME PS_TestLobTable; DROP PARTITION FUNCTION PF_TestLobTable; DROP TABLE TestLobTable; CREATE TABLE dbo.TestLobTable ( c1 int identity, c2 char(8000) default 'this is a test', c3 varchar(max) NULL ) -- will be created on FG1 go INSERT INTO dbo.TestLobTable ( c2, c3 ) VALUES ( 'this is a test', REPLICATE (convert(varchar(max), 'ABC'), 8000) ) go 1000 CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) go sp_blitzIndex @databaseName = 'TestLOB', @schemaName = 'dbo', @tableName = 'TestLobTable' GO SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.125000 FG1File1 5120 40.000000 19.687500 FG1File2 5120 40.000000 19.687500 FG2File1 5048 39.437500 0.062500 FG2File2 5040 39.375000 0.062500 --> data on filegroup 1 again... move them to filegroup2 this time with SORT_IN_TEMPDB */ /* moving Lob data using partitioning trick */ CREATE PARTITION FUNCTION PF_TestLobTable (int) AS RANGE RIGHT FOR VALUES (0) go CREATE PARTITION SCHEME PS_TestLobTable AS PARTITION PF_TestLobTable TO ( fg2, fg2 ) go CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON, SORT_IN_TEMPDB = ON ) ON PS_TestLobTable (c1) go -- removing partitioning on table CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON, SORT_IN_TEMPDB = ON ) ON [fg2] go -- now try to shrink with truncateonly DBCC SHRINKFILE('FG2File1', 20,TRUNCATEONLY); DBCC SHRINKFILE('FG2File2', 20,TRUNCATEONLY); SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 5376 42.000000 19.687500 FG2File2 5376 42.000000 19.687500 --> no significant effect...still almost 50% free space */ --- retry with separate filegroup CREATE DATABASE [TestLOB] CONTAINMENT = NONE ON PRIMARY ( NAME = N'TestLOBPrimary' , FILENAME = N'U:\DB_DATA\TestLOBPrimary.mdf' , SIZE = 100MB , FILEGROWTH = 1024KB ), FILEGROUP [FG1] ( NAME = N'FG1File1' , FILENAME = N'U:\DB_DATA\FG1File1.ndf' , SIZE = 40MB , FILEGROWTH = 20480KB ), ( NAME = N'FG1File2' , FILENAME = N'U:\DB_DATA\FG1File2.ndf' , SIZE = 40MB , FILEGROWTH = 20480KB ), FILEGROUP [FG2] ( NAME = N'FG2File1' , FILENAME = N'U:\DB_DATA\FG2File1.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), ( NAME = N'FG2File2' , FILENAME = N'U:\DB_DATA\FG2File2.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), FILEGROUP [tempLOB] ( NAME = N'tempLOB1' , FILENAME = N'U:\DB_DATA\templob1.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ), ( NAME = N'tempLOB2' , FILENAME = N'U:\DB_DATA\templob2.ndf' , SIZE = 20MB , FILEGROWTH = 0MB ) LOG ON ( NAME = N'TestLOBLog' , FILENAME = N'U:\DB_DATA\TestLOBLog.ldf' , SIZE = 10MB , FILEGROWTH = 10MB) GO USE TestLOB go ALTER DATABASE TestLOB MODIFY FILEGROUP FG1 DEFAULT go --DROP TABLE TestLobTable; CREATE TABLE dbo.TestLobTable ( c1 int identity, c2 char(8000) default 'this is a test', c3 varchar(max) NULL ) -- will be created on FG1 go INSERT INTO dbo.TestLobTable ( c2, c3 ) VALUES ( 'this is a test', REPLICATE (convert(varchar(max), 'ABC'), 8000) ) go 1000 CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) go sp_blitzIndex @databaseName = 'TestLOB', @schemaName = 'dbo', @tableName = 'TestLobTable' go -- size is roughly 40 MB: 1,000 rows; 39.2MB; 31.3MB LOB SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 2.875000 FG1File1 5120 40.000000 19.687500 FG1File2 5120 40.000000 19.687500 FG2File1 2560 20.000000 0.062500 FG2File2 2560 20.000000 0.062500 --> 2*19,687 MB are occupied in Filegroup 1 ---> approx 40 MB in total in Filegroup 1 */ /* moving Lob data using partitioning trick */ CREATE PARTITION FUNCTION PF_TestLobTable (int) AS RANGE RIGHT FOR VALUES (0) go CREATE PARTITION SCHEME PS_TestLobTable AS PARTITION PF_TestLobTable TO ( tempLOB, tempLOB ) go CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON) ON PS_TestLobTable (c1) go SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 2560 20.000000 0.062500 FG2File2 2560 20.000000 0.062500 tempLOB1 2560 20.000000 19.687500 tempLOB2 2560 20.000000 19.687500 --> 2*19,687 MB are occupied in Filegroup 1 ---> approx 40 MB in total in Filegroup 1 */ CREATE UNIQUE CLUSTERED INDEX TestLobTableCL ON TestLobTable (c1) WITH (DROP_EXISTING = ON) ON [FG2] go SELECT f.name AS [filename], fu.total_page_count AS [pageCount], fu.total_page_count/128.0 [sizeMBTotal], (fu.total_page_count - unallocated_extent_page_count) /128.0 [sizeMBUsed], fu.* FROM sys.dm_db_file_space_usage fu INNER JOIN sys.database_files f ON fu.file_id = f.file_id INNER JOIN sys.filegroups fg ON fu.filegroup_id = fg.data_space_id go /* filename pageCount sizeMBTotal sizeMBUsed TestLOBPrimary 12800 100.000000 3.062500 FG1File1 5120 40.000000 0.062500 FG1File2 5120 40.000000 0.062500 FG2File1 2560 20.000000 19.625000 FG2File2 2560 20.000000 19.750000 tempLOB1 2560 20.000000 0.062500 tempLOB2 2560 20.000000 0.062500 --> data successfully moved to fg2, tempLOB empty */ DROP PARTITION SCHEME PS_TestLobTable DROP PARTITION FUNCTION PF_TestLobTable ALTER DATABASE TestLOB REMOVE FILE tempLOB1; ALTER DATABASE TestLOB REMOVE FILE tempLOB2; ALTER DATABASE TestLOB REMOVE FILEGROUP tempLOB; /* summary: - Moving LOB Data with the help of partitioning results in twice the space needed of the original data for temporary rebuilds. - To avoid problematic and long running shrinking of database files it's best to use a different filegroup for the first rebuild as this can be easily removed afterwards. */