1

I currently have cardano-node and db-sync running on the same machine. The machine has a max RAM capacity of 32 GB and this is barely enough. Instead of buying a new machine that supports 64 GB RAM, I could use another machine I have of 32 GB RAM, to run only db-sync, on the same LAN as cardano-node.

I'd like to hear from anyone that has run cardano-node and db-sync both on the same machine as well as on different machines on the same LAN to see if you decided it had to be both cardano-node and db-sync on the same machine because on separate machines proved too slow?

Improve this question

3 Answers 3

Reset to default
2

Yes, you can do it. Run cardano-node on one of the machines and cardano-db-sync on the other, and use socat to connect from the cardano-db-sync machine to the cardano-node's socket file. This is how I am doing it.

Improve this answer
4
  • Thank you. A machine that supports 64 GB RAM is more expensive than many that only support up to 32 GB, so this being an option - using socat, is a great option. I will try it too. Right now I am running without issue since using the --disable-ledger option at db-sync startup as suggested by RdLrT.
    – RelevantData
    Nov 15, 2022 at 17:37
  • Is it fairly straightforward to use socat for this purpose? I have 2 machines, both with 32 GB RAM, both on the LAN, both running Ubuntu with cardano-node and cardano-db-sync installed and ready to be launched.
    – RelevantData
    Nov 15, 2022 at 17:39
  • 1
    Yes, it is very simple.On the machine where the cardano-node is running, you run this: /usr/bin/socat TCP-LISTEN:3081,reuseaddr,fork UNIX-connect:/path/to/your/node.socket On the machine with the cardano-db-sync, you run: socat UNIX-LISTEN:/run/cardano-node.socket,fork,reuseaddr,unlink-early,user=<your_user>,group=<your_group>,mode=755 TCP:<cardano-node-ip>:3081 And you will have your socket as /run/cardano-node.socket (you can set any path you like). Use the user and the group you are running cardano-db-sync with. You can also use any port you want.
    – George - APEX Stake Pool
    Nov 16, 2022 at 9:57
  • Thank you for these detailed instructions! Very helpful!
    – RelevantData
    Nov 22, 2022 at 2:08
2

A very big chunk of memory usage for dbsync comes from ledger-state - you can disable this component if you do not require things like epoch_stake, ada_pots, etc. You can read more about it here.

If you do require these, 48GB RAM IMO is bar minimum, and 64GB is ideal (for instance, if you optimise Postgres configurations to serve more processing capacity and less IO dependency)

Improve this answer
5
  • Was not aware of those options, I shall examine that, ty!
    – RelevantData
    Nov 14, 2022 at 5:35
  • "migrating to a no ledger schema" was reported after relaunching db-sync with the --disable-ledger flag added!
    – RelevantData
    Nov 14, 2022 at 5:45
  • 1
    Appears to work like a charm @RdLrt = thank you!
    – RelevantData
    Nov 14, 2022 at 6:46
  • 1
    That is excellent advice mate. Absolutely beautiful!!!
    – Patrick Sturm
    Nov 21, 2022 at 19:32
  • 1
    I am doing this and the memory usage is MUCH less - thank you.
    – RelevantData
    Nov 22, 2022 at 2:08
1

I'm running cardano-db-sync on the same machine of cardano-node. The machine is also a block-producer (--nonmoving-gc), cardano-wallet provider, cncli sendtip provider. The amount of RAM needed ended up to be 64GB.

The main reason driving me to host everything on the same machine is that it's a virtual machine that I can fine-tune whenever I want.

Also, you should consider this: cardano-db-sync needs a --socket-path path specified. That means that you can't straightforwardly run cardano-node and cardano-db-sync on different machines. You can probably play around with netcat to redirect I/O to/from the socket to/from a TCP socket, or mount remotely the path where the socket is located.

Improve this answer
1
  • Thank you - so far the clear winner is still "use a machine that has 64 GB of RAM"
    – RelevantData
    Nov 13, 2022 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.