How do each differ in the way that they are used?
Middleware is logical Software System that provides capabilities to other software applications, databases etc to be integrated. Middleware is analogous with Integration Software. It comes with defined capabilities like communication protocols, data transformation and mapping, connectivity etc. It also bridges physical systems and networks.
API, on the other hand is Software Architectural paradigm or a programming "style" for standardizing end-point connectivity of Systems, Applications and Data Integration.
In short, Middleware is software logical layer for Integration, while API is dominant trending/recent style/paradigm for Integration.
They go together - an API is functional definition of an Integration endpoint, while API Gateway Server is a middleware runtime component where APIs are run/hosted to be consumed by requesting applications and systems. Also API Portal is another middleware component that acts as a directory for APIs enabling their development life-cycle.