一次由SELinux引起的SSH公钥认证失败问题

一直使用CentOS作为服务器系统，平时装完系统以后都是建立publickey认证机制，然后关闭密码认证。原本是一件轻车熟路毫无压力的事情，不想前日新装一台机器按照正常配置以后居然使用publickey方式认证不成功，但是使用密码认证是可以的。 

 

具体现象表现为使用SecureCRT登陆时，出现如下出错提示：

 

Public-key authentication with the server for user sw failed. Please verify username and public/private key pair.

 

查看服务器日志，找不到有用的相关记录。 

 

之后直接在另一台机器上使用ssh连接，打开verbose模式（ssh -vvv），如下：

 

debug1: SSH2_MSG_SERVICE_ACCEPT received

 

debug2: key: /home/sw/.ssh/identity (0xXXXXXXXXX)

 

debug2: key: /home/sw/.ssh/id_rsa ((nil))

 

debug2: key: /home/sw/.ssh/id_dsa ((nil))

 

debug3: Wrote 64 bytes for a total of 1109

 

debug1: Authentications that can continue: publickey,password

 

debug3: start over, passed a different list publickey,password

 

debug3: preferred publickey,keyboard-interactive,password

 

debug3: authmethod_lookup publickey

 

debug3: remaining preferred: keyboard-interactive,password

 

debug3: authmethod_is_enabled publickey

 

debug1: Next authentication method: publickey

 

debug1: Offering public key: /home/sw/.ssh/identity

 

debug3: send_pubkey_test

 

debug2: we sent a publickey packet, wait for reply

 

debug3: Wrote 512 bytes for a total of 1621

 

debug1: Authentications that can continue: publickey,password

 

debug1: Trying private key: /home/sw/.ssh/id_rsa

 

debug3: no such identity: /home/sw/.ssh/id_rsa

 

debug1: Trying private key: /home/sw/.ssh/id_dsa

 

debug3: no such identity: /home/sw/.ssh/id_dsa

 

debug2: we did not send a packet, disable method

 

debug3: authmethod_lookup password

 

debug3: remaining preferred: ,password

 

debug3: authmethod_is_enabled password

 

debug1: Next authentication method: password

 

sw@xxx.xxx.xxx.xxx's password:

 

可以看到，ssh先尝试了使用publickey进行认证，但是失败了，日志也没有显示相关原因，然后降级到使用密码认证。 

 

接下来，又将服务器sshd的日志级别设置为DEBUG3（修改/etc/ssh/sshd_config中的LogLevel），期望得到有用的信息。修改以后重启sshd，然后再尝试连接，查看/var/log/secure，得到如下日志信息：

 

Mar 6 16:42:14 data sshd[1517]: debug1: trying public key file /home/sw/.ssh/authorized_keys

 

Mar 6 16:42:14 data sshd[1517]: debug1: restore_uid: 0/0

 

Mar 6 16:42:14 data sshd[1517]: debug1: temporarily_use_uid: 500/500 (e=0/0)

 

Mar 6 16:42:14 data sshd[1517]: debug1: trying public key file /home/sw/.ssh/authorized_keys

 

Mar 6 16:42:14 data sshd[1517]: debug1: restore_uid: 0/0

 

Mar 6 16:42:14 data sshd[1517]: Failed publickey for sw from xxx.xxx.xxx.xxx port 27816 ssh2

 

Mar 6 16:42:14 data sshd[1517]: debug3: mm_answer_keyallowed: key 0xXXXXXXXX is not allowed

 

日志信息只提到Failed publickey，并没有明确说明出错原因。 

 

检查各种配置文件，各种文件访问权限，数次折腾无果。心中郁闷，因为这台机器软硬件环境都和其它机器一样，而且SSH的publickey认证也是配置过无数次的了。最后求助万能的Google，发现serverfault上有一个案例的现象和出错信息与我遇到几乎一样，提问者怀疑是SELinux导致的。

 

案例 见 http://www.linuxboy.net/Linux/2013-07/87267p2.htm

 

下面的回复证实了确实是SELinux的问题，并且给出了解决方案：

 

Yes, SELinux is likely the cause. The .ssh dir is probably mislabeled. Look at /var/log/audit/audit.log. It should be labeled ssh_home_t. Check with ls -laZ. Run restorecon -r -vv /root/.ssh if need be.

 

 

 

Yep, SELinux was the cause: type=AVC msg=audit(1318597097.413:5447): avc:denied { read } for pid=19849 comm="sshd" name="authorized_keys" dev=dm-0 ino=262398 scontext=unconfined_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:admin_home_t:s0 tclass=file

 

It works after running "restorecon -r -vv /root/.ssh". Thanks a lot.

 

我如获救命稻草，马上用ls -laZ检查了一下我的.ssh目录，果然不是ssh_home_t，心中窃喜，立刻使用restorecon对.ssh目录的context进行了恢复。 

 

再次尝试进行连接，结果还是不行，现象和出错信息与之前一样。于是我查看了其它机器上的.ssh目录的context，都没有标为ssh_home_t，但是那些机器上的SSH服务都是正常的。我又仔细看了一下网上那个案例的描述和错误信息，我还是怀疑是SELinux导致的。于是我想把SELinux暂时关了试试，使用setenforce 0把SELinux关闭，重新尝试连接，publickey认证正常了。 

 

确认了是SELinux引发的问题，接下来我查看了/var/log/audit/audit.log，发现有如下日志：

 

type=AVC msg=audit(1362560807.992:320): avc: denied { search } for pid=1595 comm="sshd" name="/" dev=sda3 ino=2 scontext=unconfined_u:system_r:sshd_t:s0-s0:c0.c1023 tcontext=system_u:object_r:file_t:s0 tclass=dir

 

这条日志与网上案例唯一不同的地方在于案例中是sshd对分区dm-0中的authorized_keys文件没有read权限，而我的机器上是sshd对分区sda3的根没有search权限。 

 

确认了问题所在，我仔细回忆了系统的安装过程与其它机器有什么不同之处。日志中提到的sda3是系统的/home分区，当时装系统的时候由于操作失误/home分区只有200M，装完系统以后发现了这个问题，于是我把sda3分区删除重建，然后挂载到/home。这么一折腾，/home目录上的context就不对了。

 

之后我对/home目录的context进行恢复：

 

[root@data ~]# restorecon -r -vv /home/ 

restorecon reset /home context system_u:object_r:file_t:s0->system_u:object_r:home_root_t:s0 

restorecon reset /home/lost+found context system_u:object_r:file_t:s0->system_u:object_r:lost_found_t:s0 

restorecon reset /home/sw/.pki context unconfined_u:object_r:user_home_t:s0->unconfined_u:object_r:home_cert_t:s0 

restorecon reset /home/sw/.pki/nssdb context unconfined_u:object_r:user_home_t:s0->unconfined_u:object_r:home_cert_t:s0

 

然后setenforce 1打开SELinux，重新连接SSH，认证成功，问题解决。

 

SELinux是Linux的MAC（强制访问控制）系统实现，我还不太了解，看来要学习一下相关知识了。

 

public key authentication fails ONLY when sshd is daemon

 

(This question had been erroneously asked in stack overflow -- I'm sorry for that -- and closed as "off topic")

 

I have no clue on how this happens. The distro is Scientific Linux 6.1 and everything is set up to perform authentication via public key. Yet, when sshd is running as a daemon (service sshd start), it doesn't accept public keys. (To obtain this piece of log, I've changed the sshd script to add the -ddd option)

 

debug1: trying public key file /root/.ssh/authorized_keys

debug1: restore_uid: 0/0

debug1: temporarily_use_uid: 0/0 (e=0/0)

debug1: trying public key file /root/.ssh/authorized_keys2

debug1: restore_uid: 0/0

Failed publickey for root from xxx.xxx.xxx.xxx port xxxxx ssh2

debug3: mm_answer_keyallowed: key 0x7f266e1a8840 is not allowed

debug3: mm_request_send entering: type 22

debug3: mm_request_receive entering

debug2: userauth_pubkey: authenticated 0 pkalg ssh-rsa

debug3: Wrote 64 bytes for a total of 1853

debug1: userauth-request for user root service ssh-connection method publickey

debug1: attempt 2 failures 1

If sshd is run in debug mode (/usr/sbin/sshd -ddd), authentication works like a charm:

 

debug1: trying public key file /root/.ssh/authorized_keys

debug1: fd 4 clearing O_NONBLOCK

debug1: matching key found: file /root/.ssh/authorized_keys, line 1

Found matching RSA key: d7:3a:08:39:f7:28:dc:ea:f3:71:7c:23:92:02:02:d8

debug1: restore_uid: 0/0

debug3: mm_answer_keyallowed: key 0x7f85527ef230 is allowed

debug3: mm_request_send entering: type 22

debug3: mm_request_receive entering

debug3: Wrote 320 bytes for a total of 2109

debug2: userauth_pubkey: authenticated 0 pkalg ssh-rsa

Postponed publickey for root from xxx.xxx.xxx.xxx port xxxxx ssh2

debug1: userauth-request for user root service ssh-connection method publickey

debug1: attempt 2 failures 0

Any ideas?? Has anyone seen anything like this?

 

Notes:

 

File permissions have been double checked:

 

# ll -d .ssh

drwx------. 2 root root 4096 Oct 14 10:05 .ssh

# ll .ssh

total 16

-rw-------. 1 root root  786 Oct 14 09:35 authorized_keys

-rw-------. 1 root root 1675 Oct 13 08:24 id_rsa

-rw-r--r--. 1 root root  393 Oct 13 08:24 id_rsa.pub

-rw-r--r--. 1 root root  448 Oct 13 12:51 known_hosts

I was asked if sshd can access root's files in "daemon mode". The closest answer I get to this question is:

 

# netstat -ntap | grep 22

tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      19847/sshd 

# ps -ef | grep 19847

root     19847     1  0 09:58 ?        00:00:00 /usr/sbin/sshd

If sshd is running as root, I don't know how it's not possible to access its own files. Could SELinux be the cause?