The first and main rule - imgproxy responses should be cached as it doesn't have an internal cache. A couple of 1CPU 512MB servers + CDN + metrics is a good starting point. E.g., CDNs like AWS CloudFront work perfectly with imgproxy.
As for the server itself, imgproxy scales better horizontally than vertically. A swarm of single-core instances is better than one big instance. The default
IMGPROXY_CONCURRENCY value is perfect in most cases, we do not recommend changing it until acquiring several metrics. Memory consumption highly depends on the images and the way imgproxy is used. The rough prediction: 512MB should be enough unless there is an especially high number of big images processed.
See full guide in our documentation