I've a 2004 Toyota Sienna. We have 2 working remotes, and cannot program the other 2. Any tips?

We got the van at an auction, and it came with 1 working remote. We ordered 2 more, and were able to program 1 successfully. We confirmed that the battery on the second one works, and ordered another to replace the nonworking remote, but we can't program that one either. The owner's manual says we should be able to program up to 4 remotes.