Kaysar is saying he thinks the 'friendship' will get rid of James. He won't be able to say "I knew they were going to do this" as he walks out the door, because we are all witnesses that he doesn't know.
Didn't Kaysar, Rachel or Howie catch James when he was talking about the sequester house as if he knew he was going there?
NOW, Rachel tells them what James was saying. Might have been nice if they'd known before hanging out with him all afternoon. Wake up, guys!