1

I had two cardano-node instances running via kvm on one physical machine. Both nodes were running on mainnet for a while before I switched them over to testnet (to prepare for turning these into a stake pool).

I gave both environments (labelled BlockProducer and Relay) the same resources, incl. 150 GB of disk space. hard disk space for virtual environments

When logging into both environments, I noticed that BlockProducer's cardano-node process crashed with a FileSystemError, stating that there is no space left on the device. Restarting the process (also after rebooting BlockProducer) did not change this.

FsDeviceFull

These are the outputs for df -h and du -h -d 1 in the respective cardano-node directories. Left is the (working) Relay, while the right side shows the BlockProducer environment.Relay and BlockProducer comparison

The host system running the kvm daemon still has 210 GB of available disk space. Did I simply under-provision the virtual disks or what is going on here?

Why is Relay still running though? I installed the same software and configured both environments the same.

Improve this question

2 Answers 2

Reset to default
1
  1. Disk space issue:

Each system only has 73GB maximum on / (which is where you are running from). The live Cardano chain is now 50GB on it's own. I'm not sure of the testdb size.

Make the / volume larger on each system eg 120GB maybe.

  1. Differences between BP and Relay system:

The BP system has used 70GB of / while the Relay has used 65GB of /

So, what has used the extra 5GB somewhere on the BP system? Left over logs or code?

cd to /home and do a 'du -h .' there to see if you can spot if the space has been used by a different user.

If not obvious there then you will need to review other directories off of / to narrow down where the extra 5GB has been used. Often /var/log can use a lot of disk space logging things. Also check /tmp

Improve this answer
2
  • That's the thing though, I provisioned 150 GB for each of the environments.
    – Matthias Sieber
    Commented Apr 21, 2022 at 6:25
  • Yes, the whole environment may be 150G, but as @MC_Brisbane points out, the partition where the node is storing its data is smaller.
    – Erik de Castro Lopo
    Commented Apr 21, 2022 at 6:33
1

You need to find out which files are filling up the disk.

A command like:

find / -type f -printf "%15s    %p\n" | sort -nr | head -30;

might help.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.