We say at the beginning of prayer ("Ma Tovu"), Psalm 5:8 "But as for me, in the abundance of Thy lovingkindness will I come into Thy house: In Thy fear will I worship toward Thy holy temple." I understand that the first Temple was built by Solomon. Then which is this Temple David worshiped towards...? Any Biblical or historical reference or sources?

During the period of King David, the mishkan, also termed the Mikdash (Rambam, Hil. Beit HaB'chira 1, 2) was in the city of Giv'on, for about 50 years (after Nov and before Jerusalem). King Solomon goes there to sacrifice- Kings I, 3, 4; and it's there until his own Temple- Chron. I 21, 29; Chron. II, 1, 3-5; it's mentioned also in the Talmud, Zvachim 118b; Talmud Yerushalmi Megila 1, 12; Seder Olam Rabba 14.