Removing Pitcher without Facing Batter?

 
As the home team takes the field in the top of the 4th inning, the shortstop walks to the mound and begins to throw warm-up pitches (thinking he is entering the game to relieve the starting pitcher). The starting pitcher walks to the mound and believes he should still be pitching this inning. The home team coach clears things up by telling everyone that the shortstop will be pitching in the 5th inning, but the starting pitcher will continue to pitch in this inning.
 
The visiting team coach argues that shortstop took the mound and threw several warm-up pitches, so he is now the pitcher of record and must pitch to at least one batter (NF 3-1-2).
 
What does the umpire do now?
 
Call
In NF, an unreported pitching substitute is not legally in the game until he is on the pitching rubber AND the ball is made live (NF 3-1-1b).
 
In the situation below, the starting pitcher is still the pitcher of record, so he may pitch in the 4th inning, as the home team coach desired.