yesterday my one of SSDs on my server came into a faulty state. Well at least mdadm --detail /dev/md126 said that. Also mdstat showed [2/1] [_U] on one of raid partitions (md126 specifically). I couldn't create any files or edit anything because the raid was in read-only mode (for example if I tried to auto-complete anything in terminal with tab, it came up with message cannot create temp file for here-document: Read-only file system) Raid's state was clean, degraded and the sda3 partition was marked as removed and shown below as faulty.
I successfully reconnected the "faulty" partition by using these commands:
mdadm --remove /dev/md126 /dev/sda3
mdadm --add /dev/md126 /dev/sda3
The mdadm --detail /dev/md126 now shows that both partitions are active, the state is clean and everything is normal. mdstat also - [2/2] [UU].
But it seems the read-only problem is not gone. Can't create any files. Tried to remount the system with mount -o rw,remount / - no luck (mount: cannot remount /dev/md126 read-write, is write-protected). Also tried mdadm --readwrite /dev/md126 - same result (mdadm: failed to set writable for /dev/md126: Device or resource busy).
I'm out of ideas. I need to bring back full functionality without server restart.
P.s. sda drive's iowait is high, %util usually is on 100 (well it was at least before crashing). Is that a sign of dying SSD?
