1,在使用kafka0.9新的客户端消费消息时，会出现
org.apache.kafka.clients.consumer.CommitFailedException:Commit cannot be completed due to group rebalance。
的问题，出现的原因就是consumer在一定时间内没有发送心跳给coordinator（kafka 0.9后新出现的角色）。
Kafka triggers a rebalance if it doesn't receive at least one heartbeat within session time out. If the rebalance is triggered, the commit will fail. That is expected. So the question is why has the heartbeat not happened? There might be a couple of reasons for that.
First thing is that you are doing a manual commit. Starting 0.9, heartbeat doesn't happen in a separate thread. The consumer runs on a single thread which handles commit, heartbeat and polling. So the heartbeat happens when you do a consumer.poll() or consumer.commit(). So if your processing time is exceeding the session time out, that might cause the heartbeat to fail.
There is a known issue in kafka 0.9 consumer which might cause the problem you are facing.
https://issues.apache.org/jira/browse/KAFKA-3627
In either case, downgrading your consumer to 0.8 will solve the problem.
Edit: You can try increasing the session time out to as high as 5 min and see if it works.
Regarding kafka configs
Kafka server expects that it receives at least one heartbeat within the session time out. So the consumer tries to do a heartbeat at most (session time out/heartbeat times). Some heartbeats might be missed. So your heartbeat time should not be more than 1/3 of the session time out. (You can refer to the docs)
