I like Swift, too, and if it's a rap battle between Swift and Java, I think Swift has to emphasize C interop, and more recently (or perhaps "soonly"?) Python interop.
But regardless, we're gonna have to give it a few years to get a Swift ecosystem even trying to get on the mic in the same venues as Java/JDK.
I don't think this is gonna happen. There's no huge momentum behind Swift on the server. And when IBM is losing interest it will be reduced by 50 percent. Apple doesn't seem to be too much interested in Swift on a server (apart from releasing SwiftNIO, that's about it). When you take a look at the commit stats of Vapor on GitHub you will notice that there's not much going on either.
But regardless, we're gonna have to give it a few years to get a Swift ecosystem even trying to get on the mic in the same venues as Java/JDK.