1. By the rivers of Babylon, there we sat, we also wept when we remembered Zion.

2. On willows in its midst we hung our harps.

3. For there our captors asked us for words of song and our tormentors [asked of us] mirth, Sing for us of the song of Zion.

4. How shall we sing the song of the Lord on foreign soil

5. If I forget you, O Jerusalem, may my right hand forget [its skill]. 

An interesting question arises here: What possible reason could there be for the exiled Jews, who were about to embark on a walking sojourn of at least 100 kilometers from the Holy Land to Babylonia, to take harps with them? We can certainly assume that they were not provided with wagons or the like, so how is it that they thought to add harps, of all things, to their loads of a few articles of clothing, some food, and a few needed utensils on this long, sad trek? 

The answer is that they were sure that the Exile would be short-lived and that they would soon return to the Land of Israel. They knew with certainty that they would again sing "the song of the Lord" on their home soil of sanctity. They had a strong precedent for this: After the miracle of the Crossing of the Red Sea, we read of Miriam's song:

"Miriam the Prophetess, sister of Aharon, took the drum in her hand, and all the women went out after her with drums and dancing [and they sang]."

(Sh'mot 15.20-21)

Our classic Bible commentator Rashi answers the assumed question, "Why did they take drums in the first place?" and says: "The righteous women of the generation were confident [lit.: guaranteed] that G-d would make miracles for them, and they therefore took drums with them from Egypt so that they could celebrate when the time would come."

The same happened with our forefathers on their way into Exile. They knew with certainty that G-d would return them back to the Land, and that they would return once again to Zion – and so they brought harps, in order to celebrate their return. 

And if it would happen that, after a few days of Exile, they would despair – logically, they should at that point break the harps, so that they would not constantly be reminded of what they had lost. But in actuality, they did not despair; they hung the harps on trees most blatantly, so that they could be reminded not of their Exile, but of the great future in store for them.

Why does the Psalmist emphasize that they sat and wept specifically "by the rivers of Babylon"? Is that they only place in which they wept during this tragic journey? And what is the significance of the rivers?

The answer is that the reference is to the Euphrates River, which is the northern border of the promised Land of Israel (see B'reshit 15,18). Their longing to return home, even after they had already entered Babylon, brought them to actually turn back towards home and sit and weep aside the Euphrates – just like a prisoner approaches the barbed wire enclosing him, that which separates between him and freedom, and presses his nose against the bars and grasps the wire, as if trying with all his might to break through and out.

This is why the river was referred to as "of Babylon," and not of the Holy Land, even though it separated equally between both these lands – because the Jews were, sadly, on the Babylonian side, on Babylonian soil.

And so, when we recite this Psalm before Grace after [weekday] Meals, let us remember its great message: We will yet return home forever!