This is an interesting question and it depends heavily on your own goals, abilities, and resources. It's important to determine your goals because the way you prioritize various areas will shape your training routine. Up to a certain point, the best way to improve your table tennis is through table tennis-specific activities. However, working on your general fitness not only helps your table tennis, it also has benefits for everyday life. Many people do not have access to a table and training opportunities 24/7 so you should prioritize those sessions and fit the exercise into your routine in a way that gives you the most rest and ability to be fresh and focus on your table tennis sessions.
I would say it's most ideal to work out on non-playing days, second best would be later in the day or after training, or give yourself a nice rest between the gym and playing later if that is what your schedule allows. Unless you have full-time opportunities for table tennis and fitness, you will have to make some compromises for scheduling. This doesn't even take into account how much recovery you need after strenuous sessions, which varies by person. Name of the game would be starting slow and steady, and then increasing the volume/resistance/difficulty/specificity of training over time. It's better to consistently do too little, then abruptly do too much and have to take a break due to overtraining or injuries. Any extended time away from the table will hurt your progress so instead of prioritizing pushing to your limit, you would want to build in a sustainable routine that works with your table tennis training.
Then you can push harder once you are adjusted, or are going through a period when you play less. During heavy tournament months/seasons you would try to maintain the fitness level and perhaps reduce intensity/volume/frequency during those times you were focusing on your game.
For a specific but basic training mix, a mostly bodyweight routine of pushups, pullups, squats, and lunges can go a long way. These exercises incorporate a lot of body parts and help get your body recruiting muscles to get certain jobs done and it will transfer over. Jumprope and agility ladder or simply shadow practice of footwork and strokes are also very helpful away from the table. I think riding a bicycle is a great activity for table tennis players (lot of articles about it helping tennis and squash players). It can be done as easy recovery or warmup rides, along with more intense training.
Training to be strong throughout long tournament or training days is a specific goal that can be worked towards and improves mental ability during the matches and physical capacity to continue playing hard and recover as well. Hope some of that helps.