Unbreakable

No, the new covenant is not unbreakable -- at least not yet. The "new covenant" is better translated "renewed covenant" (both the Hebrew and Greek words for "new" in the phrase "new covenant" may be translated either way). The point that Jeremiah makes in Jeremiah 31:31-34 is that since the covenant has been broken, it needs to be renewed. That's why the mention of "which they broke." Nevertheless, through Jeremiah God does offer that he will forgive all his people and make sure they all know him.

We are currently in the new covenant (cf. Heb. 8), but we have not yet received all the blessings of that covenant. For example, we still have to teach people about God. It is not the case that everyone knows him, or that all the elect have been converted. Only when this happens will we be able to say that everyone knows the Lord and is forgiven. As with all the covenant blessings, we await Christ's return for the ultimate fulfillment of this great hope. At that time, Christ will purge the wicked from his people and glorify believers. This is how we will end up with a people of God in which it can be said that everyone knows the Lord.
The PCA BCO speaks of baptizing in these terms because it rightly understands that the visible church is in covenant with God and that our children are part of the visible church. When we baptize anyone, including our children, one of the things we signify is that the person baptized is in covenant with God. If that person rejects Christ, he/she is accountable to fulfill all the stipulations of the covenant on his own (one of which, of course, is to receive Christ!) -- this is the same as being without Christ outside the church, except that being in covenant with God places us in line for greater judgment. If the person baptized receives Christ, he/she is counted a covenant keeper in Christ and inherits the full covenant blessings (in due time).

Regarding the new covenant, it may be helpful to mention that the Hebrew and Greek words for "new" may also be translated "renewed." Being Reformed, I prefer this second translation because it highlights the fact that there is one covenant under various administrations rather than multiple covenants. In the Bible we see covenant renewal taking place at such times as when God reaffirms his commitment to his people and when he restores his covenant people to a good relationship with him after they have been disciplined. So in one sense, the covenant in the Old Testament was regularly being "renewed."

When Jeremiah spoke of the coming "new covenant," he was looking forward to the time when God would restore Israel to a right relationship with himself after having disciplined them in exile. When the restoration began (at the Cyrus Decree), God did begin to renew his covenant with Israel in some very significant ways: he allowed the people to return to the Promised Land; he allowed the temple to be rebuilt; he began blessing the works of their hands; he offered to install Zerubbabel as Davidic king. But because the people did not remain faithful even in the midst of this renewal process, God did not follow through with all of the blessings that he had offered to give them in the renewal. As a result, while some renewal and blessings were realized, the restoration effort ultimately failed and renewal was not finally had. The restoration continued the pattern of Israel's history: God was faithful, God was merciful, God gave and blessed; but the people rebelled and thus did not receive the full covenant blessings.

When Christ came, his work achieved what Israel's pasts works had not. He kept the terms of the covenant and thus secured full covenant renewal. The prior "renewals" had all been partial and ephemeral because the people had never managed to keep the covenant. But Christ's work was different. His obedience was complete, and his administration of the covenant is lasting. Thus, the covenant renewal under his administration will be a complete renewal. I say will be because it isn't finished yet. We have begun in earnest the final process of restoration and covenant renewal, but we have yet to realize many of the blessings offered in the new covenant (such as the resurrection of our bodies, our glorification, the final judgment and destruction of Christ's enemies, the new heavens and earth, etc.). It is right to say that we are now in the new or renewed covenant, but it is also important to realize that we still need Jesus to come back before the renewal will really be complete.