How do you debug errors with ssh-grunt?
We recently had an issue in one of our dev instances resulting in the crash of the kubelet service. When trying to do the root cause analysis we found it unusual that we couldn't access them trough ssh. Context: - We have a user in the group `ssh-grunt-sudo-users` - We have setup the ssh keys in aws - We have changed the security group to allow ssh access from the associated bastion instance. Expectations: - `ssh-grunt` runs in all ec2 instances - we would login the same way we do with other instances `user_name@instance_name` Result: - Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
There are a few possibilities here, but the best way to debug this would be to introspect the server logs. You should be able to inspect the syslog either from the EC2 console, or in CloudWatch Logs. In this scenario, I would expect there to be an error at boot time of the server that prevented it from syncing with ssh-grunt (which you should be able to see in the EC2 console, or the [cli](https://docs.aws.amazon.com/cli/latest/reference/ec2/get-console-output.html)). If you can't identify any root cause errors in there, then the next step would be to observe the system logs in CloudWatch Logs for potential error messages.