As I recall, Docker itself is a rather “hacky” solution, with lots of technical debt embedded in it. I doubt that AWS Batch itself is much better. In these domains, applying a “systemizing” approach to only your piece of the puzzle is not really feasible. If you’re really committed to the systemizing outlook, your choices are (1) yak shaving—set the original goal aside and start paying down technical debt in e.g. Docker itself until the result is simple enough to understand in a systematic way. (2) if you can’t adopt this approach, because some of the complexity you’re dealing with cannot be feasibly reduced (or you simply don’t care about doing that sort of work), then just try to abstract away what’s ‘complex’ and ‘hacky’ about the original subsystem, and build a simpler interface that suffices for yoir goals and avoids ‘leaking’ the underlying complexity as far as practicable. Both of these are in some sense high-cost, high-reward activities; hpwever, the ‘hacky’ approach comes with very real costs of its own (i.e. it provides little or no means of avoiding breakage of various sorts as complexity increases), so the common view of it as accruing technical debt that will need to be serviced or paid down later seems right to me.
As I recall, Docker itself is a rather “hacky” solution, with lots of technical debt embedded in it. I doubt that AWS Batch itself is much better. In these domains, applying a “systemizing” approach to only your piece of the puzzle is not really feasible. If you’re really committed to the systemizing outlook, your choices are (1) yak shaving—set the original goal aside and start paying down technical debt in e.g. Docker itself until the result is simple enough to understand in a systematic way. (2) if you can’t adopt this approach, because some of the complexity you’re dealing with cannot be feasibly reduced (or you simply don’t care about doing that sort of work), then just try to abstract away what’s ‘complex’ and ‘hacky’ about the original subsystem, and build a simpler interface that suffices for yoir goals and avoids ‘leaking’ the underlying complexity as far as practicable.
Both of these are in some sense high-cost, high-reward activities; hpwever, the ‘hacky’ approach comes with very real costs of its own (i.e. it provides little or no means of avoiding breakage of various sorts as complexity increases), so the common view of it as accruing technical debt that will need to be serviced or paid down later seems right to me.