SQL Server - Beberapa total yang berjalan

Saya punya tabel dasar dengan transaksi dan saya perlu membuat tabel dengan total berjalan. Saya membutuhkannya per akun dan dan juga memiliki beberapa total berjalan untuk setiap akun (tergantung pada jenis transaksi), dan di dalamnya, beberapa total berjalan per sub-akun.

Tabel dasar saya memiliki bidang-bidang ini (kurang lebih):

AccountID  |  SubAccountID   |  TransactionType  |  TransactionAmount

Mengingat saya memiliki sekitar 4 jenis total berjalan per Akun / TransactionType dan 2 total berjalan lebih per Akun / SubAccount / TransactionType, dan saya punya sekitar 2M akun dengan masing-masing sekitar 10 sub rekening, dan saya mendapatkan sekitar 10 ribu transaksi setiap menit (pada beban maksimum), bagaimana Anda melakukannya?

Ini juga merupakan keharusan bahwa ini dijalankan secara tidak sinkron melalui pekerjaan SQL, membuat agregasi tanpa menjadi bagian dari transaksi itu sendiri.

Saya cukup macet menggunakan kursor di sini - yang terlalu lama. Saya sangat menghargai saran / artikel yang melakukan kurang lebih sama.

Asynchronous menyiratkan bahwa total berjalan tidak perlu sepenuhnya akurat setiap saat, atau pola perubahan data Anda sedemikian rupa sehingga total build berjalan satu kali akan valid dan akurat hingga pemuatan berikutnya. Ngomong-ngomong, aku yakin kamu sudah memikirkan bagian itu, jadi aku tidak akan memaksakan intinya.

Pilihan utama Anda untuk metode berperforma tinggi, didukung, adalah fungsi / prosedur SQLCLR, atau UPDATEberdasarkan pada metode iterasi berbasis set Hugo Kornelis. Metode SQLCLR (diimplementasikan dalam suatu prosedur, tetapi cukup mudah untuk diterjemahkan) dapat ditemukan di sini .

Saya belum dapat menemukan metode Hugo secara online, tetapi ini terinci dalam MVP Deep Dives yang sangat baik (Volume 1). Kode contoh untuk mengilustrasikan metode Hugo (disalin dari salah satu posting saya di situs lain yang mungkin tidak Anda miliki loginnya) ditunjukkan di bawah ini:

-- A work table to hold the reformatted data, and
-- ultimately, the results
CREATE  TABLE #Work
    (
    Acct_No         VARCHAR(20) NOT NULL,
    MonthDate       DATETIME NOT NULL,
    MonthRate       DECIMAL(19,12) NOT NULL,
    Amount          DECIMAL(19,12) NOT NULL,
    InterestAmount  DECIMAL(19,12) NOT NULL,
    RunningTotal    DECIMAL(19,12) NOT NULL,
    RowRank         BIGINT NOT NULL
    );

-- Prepare the set-based iteration method
WITH    Accounts
AS      (
        -- Get a list of the account numbers
        SELECT  DISTINCT Acct_No 
        FROM    #Refunds
        ),
        Rates
AS      (
        -- Apply all the accounts to all the rates
        SELECT  A.Acct_No,
                R.[Year],
                R.[Month],
                MonthRate = R.InterestRate / 12
        FROM    #InterestRates R
        CROSS 
        JOIN    Accounts A
        ),
        BaseData
AS      (
        -- The basic data we need to work with
        SELECT  Acct_No = ISNULL(R.Acct_No,''),
                MonthDate = ISNULL(DATEADD(MONTH, R.[Month], DATEADD(YEAR, R.[year] - 1900, 0)), 0),
                R.MonthRate,
                Amount = ISNULL(RF.Amount,0),
                InterestAmount = ISNULL(RF.Amount,0) * R.MonthRate,
                RunningTotal = ISNULL(RF.Amount,0)
        FROM    Rates R
        LEFT
        JOIN    #Refunds RF
                ON  RF.Acct_No = R.Acct_No
                AND RF.[Year] = R.[Year]
                AND RF.[Month] = R.[Month]
        )
-- Basic data plus a rank id, numbering the rows by MonthDate, and resetting to 1 for each new Account
INSERT  #Work
        (Acct_No, MonthDate, MonthRate, Amount, InterestAmount, RunningTotal, RowRank)
SELECT  BD.Acct_No, BD.MonthDate, BD.MonthRate, BD.Amount, BD.InterestAmount, BD.RunningTotal,
        RowRank = RANK() OVER (PARTITION BY BD.Acct_No ORDER BY MonthDate)
FROM    BaseData BD;

-- An index to speed the next stage (different from that used with the Quirky Update method)
CREATE UNIQUE CLUSTERED INDEX nc1 ON #Work (RowRank, Acct_No);

-- Iteration variables
DECLARE @Rank       BIGINT,
        @RowCount   INTEGER;

-- Initialize
SELECT  @Rank = 1,
        @RowCount = 1;

-- This is the iteration bit, processes a rank id per iteration
-- The number of rows processed with each iteration is equal to the number of groups in the data
-- More groups --> greater efficiency
WHILE   (1 = 1)
BEGIN
        SET @Rank = @Rank + 1;

        -- Set-based update with running totals for the current rank id
        UPDATE  This
        SET     InterestAmount = (Previous.RunningTotal + This.Amount) * This.MonthRate,
                RunningTotal = Previous.RunningTotal + This.Amount + (Previous.RunningTotal + This.Amount) * This.MonthRate
        FROM    #Work This
        JOIN    #Work Previous
                ON  Previous.Acct_No = This.Acct_No
                AND Previous.RowRank = @Rank - 1
        WHERE   This.RowRank = @Rank;

        IF  (@@ROWCOUNT = 0) BREAK;
END;

-- Show the results in natural order
SELECT  *
FROM    #Work
ORDER   BY
        Acct_No, RowRank;

Dalam SQL Server 2012, Anda bisa menggunakan ekstensi fungsi windowing misalnya SUM OVER (ORDER BY).

Saya tidak yakin mengapa Anda ingin tidak sinkron, tetapi beberapa tampilan yang diindeks terdengar seperti tiket saja di sini. Jika Anda ingin SUM sederhana per beberapa grup yaitu: define running total.

Jika Anda benar-benar ingin tidak sinkron, dengan 160 baris baru per detik total berjalan Anda akan selalu ketinggalan zaman. Asynchronous berarti tidak ada pemicu atau tampilan indeks

Menghitung jumlah total yang berjalan sangat lambat, apakah Anda melakukannya dengan kursor atau gabungan segitiga. Sangat menggoda untuk melakukan denormalkan, untuk menyimpan total yang berjalan dalam kolom, terutama jika Anda sering memilihnya. Namun, seperti biasa saat Anda melakukan denormalkan, Anda perlu menjamin integritas data yang didenormalisasi Anda. Untungnya, Anda dapat menjamin integritas menjalankan total dengan kendala - selama semua kendala Anda tepercaya, semua total menjalankan Anda benar.

Juga dengan cara ini Anda dapat dengan mudah memastikan bahwa saldo saat ini (total berjalan) tidak pernah negatif - menegakkan dengan metode lain juga bisa sangat lambat. Script berikut menunjukkan tekniknya.

    CREATE TABLE Data.Inventory(InventoryID INT NOT NULL IDENTITY,
      ItemID INT NOT NULL,
      ChangeDate DATETIME NOT NULL,
      ChangeQty INT NOT NULL,
      TotalQty INT NOT NULL,
      PreviousChangeDate DATETIME NULL,
      PreviousTotalQty INT NULL,
      CONSTRAINT PK_Inventory PRIMARY KEY(ItemID, ChangeDate),
      CONSTRAINT UNQ_Inventory UNIQUE(ItemID, ChangeDate, TotalQty),
      CONSTRAINT UNQ_Inventory_Previous_Columns UNIQUE(ItemID, PreviousChangeDate, PreviousTotalQty),
      CONSTRAINT FK_Inventory_Self FOREIGN KEY(ItemID, PreviousChangeDate, PreviousTotalQty)
        REFERENCES Data.Inventory(ItemID, ChangeDate, TotalQty),
      CONSTRAINT CHK_Inventory_Valid_TotalQty CHECK(TotalQty >= 0 AND (TotalQty = COALESCE(PreviousTotalQty, 0) + ChangeQty)),
      CONSTRAINT CHK_Inventory_Valid_Dates_Sequence CHECK(PreviousChangeDate < ChangeDate),
      CONSTRAINT CHK_Inventory_Valid_Previous_Columns CHECK((PreviousChangeDate IS NULL AND PreviousTotalQty IS NULL)
                OR (PreviousChangeDate IS NOT NULL AND PreviousTotalQty IS NOT NULL))
    );
    GO
    -- beginning of inventory for item 1
    INSERT INTO Data.Inventory(ItemID,
      ChangeDate,
      ChangeQty,
      TotalQty,
      PreviousChangeDate,
      PreviousTotalQty)
    VALUES(1, '20090101', 10, 10, NULL, NULL);
    -- cannot begin the inventory for the second time for the same item 1
    INSERT INTO Data.Inventory(ItemID,
      ChangeDate,
      ChangeQty,
      TotalQty,
      PreviousChangeDate,
      PreviousTotalQty)
    VALUES(1, '20090102', 10, 10, NULL, NULL);


Msg 2627, Level 14, State 1, Line 10
Violation of UNIQUE KEY constraint 'UNQ_Inventory_Previous_Columns'. Cannot insert duplicate key in object 'Data.Inventory'.
The statement has been terminated.

-- add more
DECLARE @ChangeQty INT;
SET @ChangeQty = 5;
INSERT INTO Data.Inventory(ItemID,
  ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty)
SELECT TOP 1 ItemID, '20090103', @ChangeQty, TotalQty + @ChangeQty, ChangeDate, TotalQty
  FROM Data.Inventory
  WHERE ItemID = 1
  ORDER BY ChangeDate DESC;

SET @ChangeQty = 3;
INSERT INTO Data.Inventory(ItemID,
  ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty)
SELECT TOP 1 ItemID, '20090104', @ChangeQty, TotalQty + @ChangeQty, ChangeDate, TotalQty
  FROM Data.Inventory
  WHERE ItemID = 1
  ORDER BY ChangeDate DESC;

SET @ChangeQty = -4;
INSERT INTO Data.Inventory(ItemID,
  ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty)
SELECT TOP 1 ItemID, '20090105', @ChangeQty, TotalQty + @ChangeQty, ChangeDate, TotalQty
  FROM Data.Inventory
  WHERE ItemID = 1
  ORDER BY ChangeDate DESC;

-- try to violate chronological order

SET @ChangeQty = 5;
INSERT INTO Data.Inventory(ItemID,
  ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty)
SELECT TOP 1 ItemID, '20081231', @ChangeQty, TotalQty + @ChangeQty, ChangeDate, TotalQty
  FROM Data.Inventory
  WHERE ItemID = 1
  ORDER BY ChangeDate DESC;

Msg 547, Level 16, State 0, Line 4
The INSERT statement conflicted with the CHECK constraint "CHK_Inventory_Valid_Dates_Sequence". The conflict occurred in database "Test", table "Data.Inventory".
The statement has been terminated.


SELECT ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty
FROM Data.Inventory ORDER BY ChangeDate;

ChangeDate              ChangeQty   TotalQty    PreviousChangeDate      PreviousTotalQty
----------------------- ----------- ----------- ----------------------- -----
2009-01-01 00:00:00.000 10          10          NULL                    NULL
2009-01-03 00:00:00.000 5           15          2009-01-01 00:00:00.000 10
2009-01-04 00:00:00.000 3           18          2009-01-03 00:00:00.000 15
2009-01-05 00:00:00.000 -4          14          2009-01-04 00:00:00.000 18


-- try to change a single row, all updates must fail
UPDATE Data.Inventory SET ChangeQty = ChangeQty + 2 WHERE InventoryID = 3;
UPDATE Data.Inventory SET TotalQty = TotalQty + 2 WHERE InventoryID = 3;
-- try to delete not the last row, all deletes must fail
DELETE FROM Data.Inventory WHERE InventoryID = 1;
DELETE FROM Data.Inventory WHERE InventoryID = 3;

-- the right way to update

DECLARE @IncreaseQty INT;
SET @IncreaseQty = 2;
UPDATE Data.Inventory SET ChangeQty = ChangeQty + CASE WHEN ItemID = 1 AND ChangeDate = '20090103' THEN @IncreaseQty ELSE 0 END,
  TotalQty = TotalQty + @IncreaseQty,
  PreviousTotalQty = PreviousTotalQty + CASE WHEN ItemID = 1 AND ChangeDate = '20090103' THEN 0 ELSE @IncreaseQty END
WHERE ItemID = 1 AND ChangeDate >= '20090103';

SELECT ChangeDate,
  ChangeQty,
  TotalQty,
  PreviousChangeDate,
  PreviousTotalQty
FROM Data.Inventory ORDER BY ChangeDate;

ChangeDate              ChangeQty   TotalQty    PreviousChangeDate      PreviousTotalQty
----------------------- ----------- ----------- ----------------------- ----------------
2009-01-01 00:00:00.000 10          10          NULL                    NULL
2009-01-03 00:00:00.000 7           17          2009-01-01 00:00:00.000 10
2009-01-04 00:00:00.000 3           20          2009-01-03 00:00:00.000 17
2009-01-05 00:00:00.000 -4          16          2009-01-04 00:00:00.000 20

Disalin dari blog saya