提问者:小点点

Debezium失败-无法复制,因为主机清除了所需的二进制日志


我使用debezium-kafka来捕获MySQL中的更改。对于MySQL,我使用AWSRDS。我启用了GTID的读取副本配置。我已经将debezium与副本连接起来。

但是运行几天后,debezium错误地失败了:

org.apache.kafka.connect.errors.ConnectException: Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. To find the missing transactions, see the master's error log or the manual for GTID_SUBTRACT. Error code: 1236; SQLSTATE: HY000.
tat io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
tat io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:197)
tat io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:1018)
tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:950)
tat com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:580)
tat com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:825)
tat java.base/java.lang.Thread.run(Thread.java:834)
aused by: com.github.shyiko.mysql.binlog.network.ServerException: Cannot replicate because the master purged required binary logs. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. To find the missing transactions, see the master's error log or the manual for GTID_SUBTRACT.
tat com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:914)
t... 3 more

我无法找出导致问题的原因,MYSQL master和副本都正常,都已启动并运行。那为什么debezium失败了?


共1个答案

匿名用户

好的,我找到了解决方案,我正在使用AmazonRDSMySQL。AmazonRDS通常会尽快清除二进制日志。因此GTID在被Debezium消费之前就被清除了。

要指定RDS保留二进制的小时数:

call mysql.rds_set_configuration('binlog retention hours', 24);

链接:https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MySQL.html