Settings > Reading, set posts page, increase number of posts show.
Into a page, insert the categories basic WP element, and the posts element. Set the query on the posts element to “Current Query”. (This is why you have to set the posts page, above).
Save this as a template and delete the content from the page.
Create an archive template under Templates > Archive, insert the template you made with the template element. (You may not need to do this if you aren’t going to use it in more than one place, but if you do this is handy for consistent styling).
Set that archive to apply to both posts and categories.