I can't believe it either but even after this long, 3D printers still can't print the right size holes! After searching the web for answers I still can't find out why this is! 

So for now, what I do is print one of these templates to help me choose the right alternative dimension I need to enter into my CAD program to get the right size hole in the real world, it's an imperfect solution which works perfectly. Every 3D printer setup gives different results so if it's critical to get hole size spot on, I do this if I change settings, filaments or change the printer. I find this really useful for getting perfect clearances for nuts and bolts. I can even make a hole perfectly undersized to grip the screw threads and then not have to use a nut on the other side for example.

How to use the templates: 

Lets say you want to have a hole in your printed part to insert a nut snugly so it doesn't pop out easily. Just print one of these calibration templates that is close to the nut size. I use the 7mm Hex template for an M4 nut. Once you have the part, simply test each hole with an M4 nut until you find just the right size and make a note of which hole it is. The hole dimensions start from 7mm and go up in 0.1mm increments until 7.9mm. (measured flat side to flat side) In this case 7.8mm is the one you want. Now make the hole 7.8mm wide in your CAD design and print with the confidence that the resulting hole will be the perfect fit for the nut, whatever its real dimensions are!

Note: The holes in the part should be printed in the same orientation as the holes in the template. i.e. if you printed the template flat on the bed then the holes on your part should also be in the same plane or close at least. If you want to calibrate for holes in the XZ plane instead, then print the template standing on its side i.e. also in the XZ plane.

Hope it helps.