Creating separate small table from big based on access frequency for "select" performance

I have table with several millions of rows which are selected a lot (many times in a second in average). It is indexed and everything is ok for now. But performance is always a question (sometimes workload is too high) and I am thinking about optimization (hehe).

Among this several millions of rows, 1% of them accessed like 80% of time, and I know exactly which rows. My plan is making separate table of this rows (so it will be several thousands), I will keep theirs keywords (unique indexed) in php array (persistant cache), and every time before big table is queried, I will check this array for this keyword, and if it exists (80% of time) I will search it in the small table, if not, search in the big. Data updated not often, so it will not be hard to maintain small table. Sounds like a good plan.

The question is, is it good idea, should I try this?