Context memory retrieval tasks often implicate the left ventrolateral pFC (LVPFC) during functional imaging. Although this region has been linked to controlled semantic processing of materials, it may also play a more general role in selecting among competing episodic representations during demanding retrieval tasks. Thus, the LVPFC response during context memory retrieval may reflect either semantic processing of memoranda or adjudication of interfering episodic memories evoked by memoranda. To distinguish between these hypotheses, we contrasted context and item memory retrieval tasks for meaningful and nonmeaningful memoranda using fMRI. Increased LVPFC activation during context compared with item memory only occurred for meaningful memory probes. In contrast, even demanding context retrieval for nonmeaningful materials failed to engage LVPFC. These data demonstrate that the activation previously seen during episodic tasks likely reflects semantic processing of the probes during episodic retrieval attempt, not the selection among competing elicited episodic representations. Posterior middle temporal gyrus and the body/head of the caudate demonstrated the same selective response as LVPFC, although resting state functional connectivity analyses suggested that these two regions likely shared separate functional relationships with the LVPFC.