fill a small spray bottle with a few squeezes of dish soap, top-up with warm water, and give it a good shake
optionally (only if you don't have a cat) add a few drops of peppermint oil, so the mist also acts as a future repellent
some people also use neem oil, but I would strongly advise against this—it's toxic to mammals
you now have a basically completely shelf-stable chemical weapon sprayer, which is much better for you and your home than proper insecticides
mine's blue, and labelled with a picture of a sad wasp

there's not a great consensus as to why it kills bugs, AFAIK, but I can confirm it does indeed kill them (some types of insect need more sprays to kill them quickly, so just keep blasting until the fucker dies)
possibly it gums up their spiracles and suffocates them

also probably doesn't need to be said, but obviously never use it on harmless spiders, bees, or anything you can trap and release with a drinking glass and a piece of paper