Mysql: 大量數(shù)據(jù)批入庫耗時的問題，當數(shù)據(jù)量大時為何速度下滑很快？

java項目用的mybatis+mysql

            int batchAmount = 20000;
            for (int startIndex = 0; startIndex < size; startIndex += batchAmount) {
                int endIndex = startIndex + batchAmount;
                endIndex = (endIndex > size) ? size : endIndex;
                nInserted += userMapper.batchInsertIgnoreDuplicate(userList.subList(startIndex, endIndex));
            }

其中batchInsertIgnoreDuplicate對應的mybatis語句
Table, Columns, Values就是響應的表，表列名和列值

<insert id="batchInsertIgnoreDuplicate">
    INSERT IGNORE INTO
    <include refid="Table"/>
    <include refid="Columns"/>
    VALUES
    <foreach collection="userList" item="user" index="index" separator=",">
        <include refid="Values"/>
    </foreach>
</insert>

我發(fā)現(xiàn)入庫操作，如果數(shù)據(jù)行在百萬以及以下，每秒入庫大概2000-5000行左右
但是當數(shù)據(jù)量到了400萬左右，每秒就在400-1000行了，速度下滑顯著
我看gc回收，回收頻率和回收時間沒什么問題，400萬入庫操作時，觀察gc如下，沒啥問題，應該不是java或者jvm gc問題。那為何數(shù)據(jù)量大，入庫速度下滑嚴重？

    [root@localhost bin]# ./jstat -gcutil 10442 10000
      S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
     24.04   0.00  63.84  49.04  97.65  95.86    580   11.725     6    0.351   12.076
      0.00  23.32  71.10  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.10  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.10  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.10  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.10  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.11  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.11  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.11  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.11  49.04  97.65  95.86    581   11.731     6    0.351   12.082
      0.00  23.32  71.11  49.04  97.65  95.86    581   11.731     6    0.351   12.082
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087
     12.85   0.00  78.61  49.04  97.65  95.86    582   11.736     6    0.351   12.087