The Peter Jackson "Let it Be" documentary. They worked on music with lyrics absent, in fragments, unfinished and/or unpolished. Clearly the music was most important. But it's not a song until it's sung. To my mind, that is axiomatic. To be a song, it must have lyrics--even if they are uber-simple and repetitive. Since your question was to and about song-writers/writing, that's the only answer that makes sense.
The most important part of songwriting is to start. If you don't start, you can't finish. The second most important part, is probably to finish. You reckon? If you don't finish, it's not finished. For me, the last part to completely come together--to finish--is most often the lyrics. No longer absent, in fragments, unfinished, or (hopefully) unpolished. I can't recall ever writing a song with "dummy/stand-in" music. Nearly everything I've written has had some degree of "stand-in/dummy" lyric before it was finished. I also can't recall a song when either was completely finished before the other was well on it's way...except certain collaborations. But I've no doubt they would have proceeded differently if we'd been in the same room.
So my "vote" is...music is boss. But music without lyrics is not song-writing.