Datastax-Enterprise

快照創建問題

  • November 7, 2014

嘗試修復密鑰空間時遇到問題:

myuser@host-1:~$ nodetool repair -pr mykeyspace
[2014-11-07 13:33:10,132] Starting repair command #13, repairing 1 ranges for keyspace mykeyspace
[2014-11-07 13:33:20,668] Repair session 3baeeb40-667a-11e4-a6e9-3562b4c30e9e for range (-3651079129967449,-2] failed with error java.io.IOException: Failed during snapshot creation.
[2014-11-07 13:33:20,668] Repair command #13 finished

當我看著system.loghost-1說:

ERROR [RepairJobTask:1] 2014-11-07 13:33:20,665 RepairJob.java (line 125) Error occurred during snapshot phase
java.lang.RuntimeException: Could not create snapshot at /X.X.X.33
       at org.apache.cassandra.repair.SnapshotTask$SnapshotCallback.onFailure(SnapshotTask.java:81)
       at org.apache.cassandra.net.MessagingService$5$1.run(MessagingService.java:344)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
       at java.lang.Thread.run(Thread.java:745)
ERROR [AntiEntropySessions:12] 2014-11-07 13:33:20,666 RepairSession.java (line 288) [repair #3baeeb40-667a-11e4-a6e9-3562b4c30e9e] session completed with the following error
java.io.IOException: Failed during snapshot creation.
       at org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:323)
       at org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:126)
       at com.google.common.util.concurrent.Futures$4.run(Futures.java:1160)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
       at java.lang.Thread.run(Thread.java:745)
ERROR [AntiEntropySessions:12] 2014-11-07 13:33:20,667 CassandraDaemon.java (line 199) Exception in thread Thread[AntiEntropySessions:12,5,RMI Runtime]
java.lang.RuntimeException: java.io.IOException: Failed during snapshot creation.
       at com.google.common.base.Throwables.propagate(Throwables.java:160)
       at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
       at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Failed during snapshot creation.
       at org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:323)
       at org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:126)
       at com.google.common.util.concurrent.Futures$4.run(Futures.java:1160)
       ... 3 more
ERROR [Thread-1607] 2014-11-07 13:33:20,668 StorageService.java (line 2599) Repair session 3baeeb40-667a-11e4-a6e9-3562b4c30e9e for range (-3651079129967449,-2] failed with error java.io.IOException: Failed during snapshot creation.
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.IOException: Failed during snapshot creation.
       at java.util.concurrent.FutureTask.report(FutureTask.java:122)
       at java.util.concurrent.FutureTask.get(FutureTask.java:188)
       at org.apache.cassandra.service.StorageService$4.runMayThrow(StorageService.java:2591)
       at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
       at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.io.IOException: Failed during snapshot creation.
       at com.google.common.base.Throwables.propagate(Throwables.java:160)
       at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
       ... 1 more
Caused by: java.io.IOException: Failed during snapshot creation.
       at org.apache.cassandra.repair.RepairSession.failedSnapshot(RepairSession.java:323)
       at org.apache.cassandra.repair.RepairJob$2.onFailure(RepairJob.java:126)
       at com.google.common.util.concurrent.Futures$4.run(Futures.java:1160)
       ... 3 more

當我查看日誌時,X.X.X.33我看不到任何日誌錯誤。

誰能給我指點正確的方向?

固定的。我發布了一份手冊nodetool snapshot mykeyspaceX.X.X.33並設法在日誌中找到了更好的錯誤。原來我有一個失去的數據文件,可能是由於刪除表並創建它的速度比架構更改在整個集群中傳播的速度更快。重新啟動 Cassandra 節點(以便它發現文件失去)並截斷表。

引用自:https://serverfault.com/questions/642722