Yeah, we're trying to solve for this over at https://vector.dev. We intend to decouple the pipeline from the method. As demonstrated by OT, this stuff changes, and you shouldn't have to rip out your plumbing every time it does. We're aiming to support all data (logs, metrics, and traces) as well as popular standards.
1) Not currently, but it's certainly possible. We don't foresee any major fundamental changes. The most likely change is around transforming. Specifically, how you shape both log and metrics data structures. Right now it's a little too piecemeal and we hope to provide ways to consolidate that more. For example, we have a transform for adding, removing, renaming, and coercing fields. We'll probably introduce a single transform to support that or think about ways to support schemas.
2) Yep! We're organizing our content and education strategy to start sharing this information. Expect to see high-quality guides, blog posts, and comparisons in the next 6-12 months.
I'm biased because my team and I created Vector [0], but I'd highly recommend investing in a vendor-agnostic data collector to start. You can use this to collect your data and send it wherever you please. This will afford you the flexibility to make changes as you learn more, which will be inevitable.
Absolutely, this is likely the next integration we'll be working on. There were a few features schema-wise that we needed support before we started, but we're _very_ close. We'd love beta testers to help us build it out. Feel free to email us if you're interested: vector@timber.io
Heyyy you're the one who wrote authlogic! What a blast from the past :) Thank you for your contribution to the Rails community, and congrats on shipping Timber & Vector!
Telegraf is nicely done. We spent a lot of time testing solutions in our test harness (https://github.com/timberio/vector-test-harness) and Telegraf was the most impressive of the tools we tested, so kudos to the Influx team on that.
But to answer your question, telegraf is very heavily metrics focused, and their logging support appears to be limited (reducing logs to metrics only). Vector is _currently_ focused on logging with an eye towards metrics, but still has work to do on the metrics front.
For example, we opened the door with the `log_to_metric` transform (https://docs.vector.dev/usage/configuration/transforms/log_t...) to ensure our data model supports metrics, but we still have a lot of work to do when it comes to metrics as a whole. Our end goal is to eventually replace telegraf and be a single, open, vendor neutral solution for both logs and metrics.
It's very likely we're doing something wrong with this test, but after many hours of trying we couldn't get our simple test to pass for Logstash, even though it passed for others:
Also, thanks to github, if a plugin is popular enough it never really gets abandoned. Look at the engines plugin. The author of that plugin rarely does anything to it, but people fork it and keep it up-to-date.
