I believe FreeCAD assumes imported files are in cm regardless what the units are set to.
Begin by using a scaling factor of 2.54 (2.54 cm / 1 in).
Then multiply the scaling factor by the factor shown on the drawing. For example, if a scale of 2:1 then multiply by 2 (2.54 x 2 = 5.08).
If that doesn't work than the scale on the drawing is probably incorrect. Measure a distance on the imported DXF file and compare to the drawing and adjust the scaling factor accordingly. For example, a scaling factor of 5.08 gives 1.35" instead of 1.5", multiply the scaling factor by 1.5 / 1.35 = 1.11 should result in the proper dimensions.