Error goes like this – Failed to perform requested operation on instance “TestUbuntu2”, the instance has an error status: Please try again later [Error: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7bf9297f-bcdc-4c4a-a425-a70d2c4739d3.].
I logged into my block storage node (cinder) and went to see logs /var/log/volume.log
Everything was completely ok
After that I went to compute node (nova) and again, went to see logs /var/log/nova/nova-compute.log
There were quite a few warnings and errors
INFO os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] Trying to connect to iSCSI portal 10.0.0.41:3260
2020-04-24 08:53:22.602 1962 WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] Failed to login iSCSI target iqn.2010-10.org.openstack:volume-325b8c0f-4c52-470f-bd76-7de58c8f586c on portal 10.0.0.41:3260 (exit code 8).: ProcessExecutionError: Unexpected error while running command.
2020-04-24 08:55:22.710 1962 WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] Failed to connect to iSCSI portal 10.0.0.41:3260.
2020-04-24 08:55:22.721 1962 WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] Couldn’t find iSCSI nodes because iscsiadm err: iscsiadm: No records found
: VolumeDeviceNotFound: Volume device not found at .
2020-04-24 08:59:29.027 1962 WARNING os_brick.initiator.connectors.iscsi [req-172c4913-748f-4892-b951-2c6f115e26e7 e9bf7a3b5599497ab3e6c145094b1b4f 339f310a18084fc1a4a3b4b85b27c516 – default default] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Ok, I did my whole lab with firewall on, and SELinux enabled – that is not recommendation in official documentation.
Again, I did it, because I wanted to learn as much as I can, and again, when I put OpenStack in production it is secure solution.
So, on my block storage node 10.0.0.41 (cinder) I failed to open port 3260
Lets check firewall opened ports on cinder node
firewall-cmd --list-all
Ok, so no opened ports
sudo firewall-cmd --add-port=3260/tcp --permanent
sudo firewall-cmd --reload Again, I checked for opened ports
After that I tried to create new instance with CirrOS image
Success
I also encountered this eror while installing PackStack on CentOS – if you installed PackStack on fresh, clean CentOS (8) machine, maybe you will stil get this error. In case of PackStack I got this error if the SELinux is in enforcing state. It gave me errors on NOVA, again in iSCSI – access denied.
The exact error was
"Permission deniedd\niscsiadm: Error while adding record: encountered iSCSI database failure\n'"
SELinux was the problem.
Enter
vi /etc/selinux/configSet SELINUX line to following:
SELINUX=permissiveReboot your PackStack install and try again.
So, in my case firewall was the problem, and on PackStack I had SELinux enforcing. It may be different in your case. I wanted to show you procedure for troubleshooting more or less everything in OpenStack – check logs first and try to go through them, there should be your problem and potentially – solution.