It may be attached in the trunk or you snap off the lens from inside the car, but you know it has to come off somehow! Buy a http://www.haynes.com manual for your car, but it probably won't help. Join a Toyota Lexus online owners group and read all the owner comments. You'll find a tip sooner or later.
http://autorepair.about.com/cs/troublesh