The extent to which a drug inhibits a target responsible for a therapeutic effect is a more rational primary endpoint for dose-finding studies of more selective anticancer drugs than the conventional endpoint of dose-limiting toxicity (DLT) used for cytotoxic agents. An adaptive phase I trial design incorporating maximum target inhibition as the primary endpoint was developed to define the optimal dose of talabostat, a dipeptidyl peptidase (DPP) inhibitor, in children with relapsed or refractory solid tumors. The relationship between dose and effect (percent inhibition of serum DPP-4) was assessed using a maximum effect model. Maximum target inhibition was defined as greater than 90% DPP-4 inhibition in five or more of six patients 24 hours post-dose. If DLT was to occur, the trial would adapt to a traditional phase I design with a more conservative dose escalation. At the 600 microg/m(2) dose level, serum DPP-4 inhibition at 24 hours was 85%. No talabostat-related DLT occurred. The maximum effect model predicted that 1200 microg/m(2) of talabostat would maximally inhibit DPP-4. This adaptive trial design appears to be feasible, safe, and efficient and warrants further evaluation for development of molecularly targeted agents.