Two-body abrasive wear tests of Fe-B alloys with various matrix microstructures were performed using a pin-on-disc tribometer at a normal load of 3 N. The wear behavior was analyzed using scanning electron microscopy (SEM) and color 3D laser scanning microscopy. The results show that the Fe-2 wt% B alloy is mainly composed of a metallic matrix, M2B and M-23(C, B)(6). A pure pearlitic matrix occurs at a cooling rate of 0.05 degrees C/s, and a pure martensitic matrix forms at a cooling rate above 0.3 degrees C/s. Compared to the pearlitic matrix, the martensitic matrix can better support the M2B against fracture and provides higher abrasion resistance for the Fe-2 wt% B alloy. Moreover, with an increase in sliding distance, the abrasion resistance of Fe-2 wt% B alloy decreases slightly at first and then decreases rapidly. To be exact, the M2B is steadily scraped off until the critical sliding distance of 6.06 m is reached, after which the neighboring M2B fractures, leading to high material removal.