Among all Banach spaces which are isometrically invariant under time-frequency shifts there is a minimal one, the modulation space $M^{1,1}$, also called the Segal algebra $S_0(R^d)$. It can even be defined over general LCA Groups
by the property of its members of having an integrable short-time Fourier transform. Essentially it means that the usual $L^2$-condition is replaced by the slightly stronger $L^1$-condition.

Together with the Hilbert space $L^2(G)$ and the dual space $S_0'(G)$ they form a so-called Banach Gelfand Triple which allows to describe things like the Fourier transform, the kernel theorem etc. in a good, functional analytic way. The ``good elements'' allow to just use the Riemannian integral, the Hilbert space in the middle allows to describe the orthogonality (unitarity) of transforms, while the outer layer (distributions) allows to describe the true behaviour (i.e. the Fourier transform mapping pure frequencies into pure point measures).