Rocksmith+ Dev Diary – February, 2022: Audio Updates

During the Rocksmith+ beta we received invaluable feedback from you, our users, on the state of the audio and note detection side of the Rocksmith experience.
Some of this feedback was positive, some was critical, and we made sure to take all of it to heart. We pored through the Discord and forum going through your feedback. The direct interaction we were able to have with many of you pointed us not only songs, but specific phrases, passages, or chords that were proving to be problematic.

To all of you participated and helped us with your feedback I want to take a moment to say thank you. It is your passion for this product that drives us forward and helps to shape the experience, we couldn’t do it without you. As such, we have continued moving forward with the things you enjoyed, and we are working hard to improve the issues that were surfaced. The community has directly impacted our priorities and helped us to home in on the more subjective aspects of gameplay related to note detection as well as our newly added “other interface” support and it is on those subjects that I would like to take a deeper dive today.

The subject of chords is a really interesting one. We have worked on them a lot (please don’t tell production how much) to make significant improvements over how chords were detected in Rocksmith 2014. There is a fine line that we need to walk between outright detection perfection and feel as a player. We can push the limits of the system so that we need to hear every note cleanly ring out, but the fact is, that isn’t really how we play guitar. To require that level of precision every time not only feels robotic as a player but sounds robotic too! So, while it is our goal to help guide you on your musical journey, be that teaching you from the beginning or helping you hone the skills you already have, we don’t want to lose sight of the fact that music is a performative art. Detection needs to feel good and feel accurate while encouraging you to perform and have fun, and therein lies the challenge.

This is where we have spent countless hours, staring at wiggly lines that are a visual representation of the instruments’ output. From here we can adjust a multitude of parameters to better reflect what has been played in the results that we get. Building out a robust tool set was a huge leap forward for improving note detection and being able to both see and quantify what detection is hearing.

Our goal in development was to take detection of chords as far as we could, to make it as accurate as possible so we knew what we could do and walk it back from there. As many of you experienced and commented on in the beta chords were strict. We’d walked it back a bit from the zenith of outright perfection, but many of you made it clear that we could loosen things up a bit more to achieve our goal of keeping the experience organic, one that encourages musicality over robotic perfection. So, how do we maintain accuracy while loosening up the feel?
Through a lot of work to be completely honest. You were able to give us reports pointing to phrases and specific chords that were problematic and we went through and played all those songs and hammered away at those chords. Through examining all this data we were able to determine patterns about what was and wasn’t working.

With this data in hand we able to go back and fine tune how we were picking notes in chords and the weight that those notes carry. In Rocksmith+ if we ask you to play a major chord and you play it minor; we know. This was something we struggled with in 2014, and an improvement we remained committed to keeping. Similarly, if there is a seventh in a chord, we are serious about that color tone. Often times, on guitar, a chord will have a series of unique notes as well as some that are doubled. A six-string major bar chord, for example, has three root notes, two fifths, and only one third, but it is that single third that really defines the quality of the chord. This third, along with the root note in the bass of the chord, we weigh the importance of hearing pretty heavily as they are the most defining features of that chord’s harmonic function. There is much more going on behind the curtain, but those are some of the factors we consider in balancing musicality with accuracy.

We also did a complete reevaluation of vibrato detection based on the feedback we received. We realize that vibrato is an expressive tool in your toolbox as a musician and to judge you harshly on how you are expressing yourself in the moment is not what Rocksmith+ is here to do. We were looking for something very specific in this first pass at vibrato detection and many of you were very vocal about not feeling like detection was hearing you not enjoying the experience of playing vibratos. We have redesigned the system from the ground up so you can now choose to be expressive in the way that fits your personal musicality. We hope that you enjoy it, and if you don’t, tell us! We’re listening.

Now, let’s talk a little bit about a new feature we added to Rocksmith+ “other interface” support. Supporting other interfaces on PC is a big change for us and a decision that we did not take lightly. Continuing to only support the Real Tone Cable would have been, by far, the simplest solution for us as it removes a variable that we have to develop around. We know there are a lot of places in the world where getting a Real Tone Cable falls somewhere on the spectrum between expensive and impossible, while other players already have their own hardware that they would prefer to use. We want to welcome all of you into Rocksmith+ in an official capacity with no third-party solutions necessary.

For a lot of you I know the excitement here is going to be ASIO support. In Rocksmith+ we have implemented native ASIO support, not ASIO support through any sort of Windows audio/Wasapi translator. Now there are some positives and negatives to this approach. The biggest positive is the low latency that ASIO provides, the immediacy between hitting a note on your instrument and hearing it play back that is so immersive and just feels right as a player. While we have worked hard to optimize the experience with the Real Tone Cable there will always be operating system overhead that you just can’t get around without integrating ASIO support. The potential downside to ASIO is that it is, by design, full duplex. Full duplex means that your input and your output are coming from a single device, it is even right there in the name, ASIO stands for Audio Stream Input/Output. Being sure to route all your audio through a single device is part of how ASIO achieves such low latency, so you have to decide for yourself if that is a sacrifice that makes sense for your playing experience.

Other interface support is not just limited to ASIO devices, it also works with Wasapi which is the Windows Audio Session API. Wasapi is Microsoft’s low latency audio solution for Windows. Wasapi brings the flexibility to choose nearly any input and output combination that ASIO is lacking, but at the sacrifice of the extreme low latency performance that ASIO brings to the table. This doesn’t mean that Wasapi is high latency; an optimized setup can hit a round trip latency of about 25 milliseconds. Round-trip latency is how long it takes from the time you pluck a note on your instrument to when that sound comes out of your speakers. To put this in perspective this is about the same as standing roughly 25 feet from your amplifier, readily achieved on a good-sized stage, so this is an amount of latency that you could easily experience in the real world with a completely analog setup. Wasapi support means that you can bring a different cable, if that is what is available to you, or you can even play Rocksmith+ with a USB microphone and an upright bass like the dev team has done. The Real Tone Cable is also a Wasapi device. There are a lot of options out there in the world so be sure that any hardware you want to use with Rocksmith+ supports operation at 16 bits and 48 khz.

For the beta this feature came in hot and while it worked perfectly for some others got connection errors, which is a frustrating experience. The team has put in an absolute ton of work to make this system more stable and reliable than what we were able to show in the beta last July. The engineers working on this feature have tracked down and fixed a lot of bugs and we have gone so far as to completely revamp the system that connects you under the hood. In addition to stability improvements, we’ve worked on making some feature improvements as well. Many of you noticed during the beta that you were unable to change the buffer size of your ASIO device, and I am happy to say that we now support being able to change this from inside Rocksmith+ itself. The other thing lacking in the beta was a simple way to choose your input channel for those whose instrument input was not channel one. While there was a work around for this it was complicated and obviously not how the feature was supposed to function. We now support changing your input channel from the settings for your device inside of Rocksmith+.

We have made huge strides in the functionality of our other interface support, but we do still consider this to be a beta feature at launch. While some of us have huge stacks of interfaces on our desks grazing the ceiling there are simply too many options out in the world for us to test them all. Our focus has been on the most popular devices and the ones produced by our partners, and we will be sure to provide a list of known supported devices at launch. As always, thank you for the support, and we’ll be talking to you all again soon!

-Brian Poedy,
Lead Audio Designer