Let me try to answer your questions, although it's very dependent on the actual use case on what structure for off-chain processing of these "tens of thousands transactions per second" would make sense.
For hydra-pay on the main chain, is the limitation of how many
transactions you can handle per second for your users dependent on how
many Hydra nodes you are running or what? What kind of infrastructure would > be needed for this? From my understanding, one solution for this use case is > opening up a Hydra head for every user during the time they are transacting.
hydra-pay is a tool that makes pairwise payment channels based on Hydra Heads easy to create and use. As you are bringing it up, I assume you have a use case where users want to process their transactions with a single other party, e.g. another user or your service.
Infrastructure: In such a pairwise setting, you would have two hydra-nodes running, one for each participant. For highest security, users would have their own hydra-node running on their end, e.g. wrapped up in a client-side application. When using hydra-pay though, nodes are run in a "hosted" way. That is, secrets would stay on the client-side (not yet, future feature), while the actual operation would stay with the service operator of hydra-pay.
Scalability: Now the capability of how many transactions can be processed depends on the performance of the machines running the hydra-nodes, their bandwidth, the latency between them and also of your users' machines which at the very least need to sign transactions - unless you are keeping the keys of your users (which would make the use-case custodial).
For a Cardano sidechain, what are the limitations for how many transactions > you can settle per second? Is it just based on the consensus algorithm?
For the mainchain, for a sidechain, for a Hydra Head.. for any decentralized network the limit on scalabality is a consequence of how decentralized (number of nodes, their latency & bandwidth, minimum system requirements) and how secure a network is (redundant communication, consensus algorithms, cryptographic complexity). It is also important how big and complex the transactions you care about are. All of these influence how long it takes a transaction to be processed & finalized by the network and in consequence how many transactions you can settle per second.
