Rocksmith+ Dev Diary – March 2022: Chord Charts & ARCHI

Rocksmith+ is a massive project, spanning multiple countries and hundreds of skilled, devoted team members, some of whom have jobs that are so specific that their job titles can be totally opaque at first. For example, I’m a Music Analyst. What does that mean? Well, it means I work with Chord Charts, mostly.

“What’s a ‘Chord Chart’?” Great question. They’re a new kind of arrangement in Rocksmith+, one of the biggest new features by sheer volume, and they make more songs playable for more people than ever before.

“…But what’s a Chord Chart, really? You haven’t actually explained anything.” Fair point! Here's a brief 'lightning round' explanation:

• Chord Charts are arrangements of songs instead of strict transcriptions - they attempt to capture the 'feel' of the song in its entirety rather than the exact notes of any one specific instrumental part

• Chord Charts avoid single-note lines and techniques entirely and are instead composed entirely of chords - hence the name!

• The chords used in these arrangements are drawn from a streamlined collection of chord shapes and fingerings, chosen to be approachable and repeatable for players of all skill levels despite including the crunchy 'tension notes' that make the songs sound like themselves

• Chord Charts use the same Noteway you're familiar with from our other arrangements, with new Chord Diagram features in RS Tab allowing you to customise your Chord Chart experience just the way you like it – including all our existing accessibility features, of course, and the Riff Repeater for guided practice

• Scoring accuracy in Chord Charts works slightly differently than in other arrangements: as long as the chord is played correctly at least once between the start and the end of the Chord Zone it'll register as correct, no matter what else you play or don’t play. Chord Charts are meant to be more expressive, which is why we’ve adopted a more lenient approach.

• Chords played in Chord Chart arrangements contribute to your overall Skill Progress as a player, helping you assess which chords you’re best at and which need work across songs, even including Authentic Arrangements

• All Chord Charts are in E Standard tuning and don’t require a capo, even if the Authentic arrangements for that song are in alternative tunings and/or use a capo. For Chord Charts, as long as you've got all the strings on your guitar, you're probably good to go!

Whereas a Lead, Rhythm or Bass arrangement might not be recognizable as the song if you play it on its own without accompaniment from Rocksmith+, a Chord Chart is an arrangement you can learn, then play even without RS+ and still capture the essence of the song. You may be familiar with these types of arrangements described as a campfire arrangement, a hum and strum version of a song, or the kind of performance you might see from an acoustic cover. Within RS+ on the other hand, Chord Charts are not only a way to make our entire library accessible to guitarists at all experience levels but also low-intensity entry points to songs whether you know and love them already or are hearing them for the first time. (Personally, I’ll often start playing a new song with the Chord Chart, as it lets me get a sense of which arrangement I’d like to learn.)

Shapes and fingerings learned in one Chord Chart will apply directly to all other Chord Charts, as they all draw on the same ‘vocabulary’. So the more of them you play, the easier you’ll find them, even if you’re just starting your guitar journey. Chord Charts also use a unique Difficulty Ranking system, since the skills used for Chord Charts are different from those of other arrangements. And thanks to the more permissive scoring system found in Chord Charts, beginners can focus on getting their fingers around the chord changes, while advanced players can choose to practice their strumming and arpeggios or put their own spin on the song by adding riffs and licks between chords.

That’s the beauty of Chord Charts. They're unique arrangements for players of all skill levels that lower barriers of entry to our song catalogue. Chord Charts teach you the ‘essence’ of a song that much quicker and lets you focus on the skills to hone your personal playing style.

But what’s most remarkable about Chord Charts in RS+ is that every single song in our catalogue will have a Chord Chart arrangement available for you to play. If you see a song, you can play it! And that will remain true for every song from the beginning until we hit our goal of millions of playable songs.

“Every single song in our catalogue will have a Chord Chart arrangement available for you to play.”

That might sound unbelievable, I understand! That’s a lot of songs, and every single one of those Chord Charts has hundreds of individual pieces that need to be in exactly the right place before it meets our standards. With that in mind, it can sound a little unlikely that we’ll get through a million songs.

But as I said up top, working with Chord Charts is what I do, along with all the other Music Analysts on the team-and in the long run we’re feeling good about hitting that goal. Partly that’s because Music Analysts tend to be the sort of people who’d be listening to and playing music obsessively even if you didn’t pay them, but also because we’re not the only ones working on Chord Charts. The driving force that really makes it possible is our proprietary Machine Learning algorithm which we’ve called the Automated Rocksmith Chord Interpreter; or A.R.CH.I. for short. (I’ll call it ‘ARCHI’ from here on.)

ARCHI

Now I can’t speak to the finer points of ARCHI, as the real brains behind the operation are our Data Science team and their research partner, a leading researcher in the field, but I can tell you how the Music Analyst team uses it. From how it looks here in the Music Analyst Team, ARCHI listens to a song once, takes a rough guess as to how our finished work will look, and gives us an unverified version. In that process, ARCHI not only evaluates what the Chords of the song should be and how to make them playable, but also identifies the placement of every beat in the song in relation to the audio we use as a basis. That’s hundreds of small tasks for each song that we’d otherwise need to manually take care of one at a time, and ARCHI can complete them in less time than it would take us to even listen to the song once! After ARCHI produces its first version of a Chord Chart the Music Analyst team reviews its work and makes any refinements or changes as needed so that it can become “Verified.”

You might choose to think of this as like having a friend who can help you cheat on the test. That’d be a fair comparison, except for two points:

1. ARCHI’s answers are not always guaranteed to be correct; and

2. We taught ARCHI everything it knows. Literally!

That’s not to say that ARCHI is full of errors. We’ll occasionally find that ARCHI has produced a near-perfect Chord Chart all on its own, one that basically just needs its ‘I’s dotted and ‘T’s crossed. And the frequency of that is increasing as time goes on – it is still somewhat astounding to see it actually happen, but it’s becoming a regular occurrence at this point. The reason why it doesn’t happen more than that is related to point 2 above and will take a bit of explanation. I hope you’ll bear with me, as it’s a fairly abstract topic and will necessitate a few paragraphs of wading through the Conceptual Wilderness for me to communicate how it all comes together.

The Conceptual Wilderness

ARCHI, and machine learning algorithms as a whole, are ‘trained’ on a regular basis. What that means is, we show it all of the Chord Charts we’ve verified, and tell it “Hey, do it more like this.” Much like a human being, it will follow those instructions in many ways, and sometimes not the ones you intend. But compared to a human being, who might get specific training on how to write Chord Charts and past music education they can draw on, ARCHI can learn by example by considering the music that we show it. A human is taught a process to follow while ARCHI is shown a finished product and asked to get there.

So, this means that every time we complete and verify a Chord Chart, ARCHI suddenly has one more point of reference – and we’re adding new verified Chord Charts every single day. The more data ARCHI has the more informed it is. But even though the selection of songs in RS+ will of course be incredibly varied by the time we launch, it still won’t be every song in the world, at least not yet, and the total amount ARCHI will ‘know’ will probably be comparable to how many pieces of music any one of our Music Analysts knows in depth at this stage in their musical development.

While that’s very cool, and is a lot like having a clone with perfect recall on the job, it’s natural that ARCHI’s frame of reference can sometimes still be too small to know what’s best for a song, often because we haven’t given it that frame of reference yet. It also can’t really ‘analyse’ the way we often do on the job. When we’re creating Chord Charts, we spend a lot of time figuring out what is or isn’t ‘essential’ to a song, which requires a lot of real-life musical experience that’s almost sociological in nature – knowing who plays what, and how, and why, and what they think they’re playing.

For example: Different genres have different rates of ‘harmonic change’, which basically refers to how often and how fast chords are expected to change, and based on that expectation we can hear some notes changing and conclude “oh, this is really just part of the chord before it”, or alternatively we can ‘hear’ the absence of notes – when Django takes off for a solo and plays different ‘chords’ from any we’ve heard before, we can infer he’s really ‘playing’ the same chord progression we’ve already heard, but adding extensions, elaborations and passing notes on top of imaginary ‘chords’ that nobody is actually bothering to play. (You’ve already heard them once, and in jazz there’s an expectation that you’ll probably remember where they’re supposed to go after that, so there’s no need to be so tasteless as to actually play them.) And this isn’t even getting into the differences between how genres tend to choose chord shapes – we’ve had passionate discussions about how often 1st wave punk bands used full barre shapes, for example, despite what’s usually a safe assumption that ‘punk’ means power chords.

Despite the fact that ARCHI can theoretically make a list of every sound it hears in a song faster than a human could ever manage it, ARCHI can’t really ‘listen’ to songs the way we do, at least not yet, because music isn’t just about ‘hearing’ sounds. We might find ourselves ‘listening’ to a song that uses power chords, noticing the singer adds the 3rd of the chord for a brief moment, but deciding to use a power chord there anyway because it’s most consistent with what someone wants to play. ARCHI might hear the full chord which, strictly speaking, is exactly what’s happening, but only desirable if one ‘sums up’ all the audible sounds. It’s deciding whether a song is in 6/8 or 3/4, especially if there’s intentional syncopations during the song. Given a ton of information, ‘listening’ means being able to know which of that information is most valuable and which can be disregarded or de-emphasized, at least when it comes to these tasks.

There are many contextual decisions like this that ARCHI is still developing the ability to handle. If you were a brain in a vat, you’d find context tricky, too. (Not to say that many of us haven’t felt like a brain in a vat ourselves during lockdown.)

Nevertheless, there’s a sense in which these things can be learned, and we’re working to make that a reality. After Chord Charts had their ‘first contact’ moment with the public during the Open Beta, we’ve been working hard not just on tightening up the Chord Charts themselves, but also on how we refine and improve the work ARCHI is doing. Our feedback loop is faster, and beyond simply feeding ARCHI our finished work as we used to do, we’re continually establishing new patterns of review and verbal feedback to help our data science wizards get the right balance between eye of newt and toe of frog. And, of course, every Chord Chart we verify is more knowledge and experience for ARCHI to draw on. How many songs do you need to understand before you theoretically have enough knowledge to understand any piece of music? We don’t know yet, but we might find out, and may even end up with an exact number or a list if we’re really lucky. (Voyager record, eat your heart out!)

“Every time we complete and verify a Chord Chart, ARCHI suddenly has one more point of reference – and we’re adding new verified Chord Charts every single day.”

By the time you get your hands on Rocksmith+ you may even find some of ARCHI’s early work listed as ‘Unverified’ Chord Charts. We’ll have these available in Rocksmith+ with each content launch just so that you can get to playing and making your own arrangements in Rocksmith Workshop for the new songs even faster—before we’re able to finish revising the Chord Charts ourselves. They will be the minority, however, and should only be around for a short time before they’re replaced by the Verified versions, if you’re able to find them at all. You’ll be able to follow ARCHI’s progress in real-time, just like we are. And one day, we’re hoping ARCHI’s work will be nearly indistinguishable from a human Music Analyst’s.

“But if the Machine Learning algorithm’s work will be indistinguishable from a human’s, and it’ll be faster than you can do it, what’s the point in keeping you around?” First of all, rude. Second of all…

All the Things ARCHI Won’t Do

There are many things ARCHI can’t do on its own at present. With some of those things, we have a reasonable expectation it’ll be able to do them on its own eventually. Others, we hope, it’ll be able to do in the future. And then there’s some that will likely need to be completed by humans no matter how advanced the system gets.

An example that goes to both extremes is Tempo Maps, which I mentioned briefly earlier. They’re one of ARCHI’s particular strong points in most cases. If a song is in 4/4 throughout, we can usually count on ARCHI being able to identify and pin down the beat with a high level of precision, saving us anywhere between minutes and hours depending on how much the drummer of the song practiced with a metronome.

But there’s a downside – at present, ARCHI tends to look for the loudest parts of the song and place the beat there. That’s great for rock but is exactly the opposite of what we’re looking for in genres that tend towards syncopation like funk and reggae, and we’ll often end up with a perfect result half a beat away from where it needs to be, ultimately requiring us to re-do the whole thing manually instead of just making small corrections. Similar problems abound with time signature changes, ‘cut’ bars and a whole panoply of related phenomena that show up in more places than you might expect. There are even some moments that will always be impossible for ARCHI, like those where experienced analysts need to make judgement calls about what’s happening rhythmically because the audio is ambiguous. (That includes situations where the band’s timing could charitably be described as ‘loose’.)

We can expect this situation to improve as ARCHI builds its knowledge and can recognize ‘genres’ better, but for now we’ve got to keep a watchful eye on our little buddy. And as you might imagine given the explanations above, that’s even more true for chords! Practically, for the near future we’ll be completing most of these Chord Charts as a collaboration with ARCHI, and much like any collaboration there’ll be many times when one of the participants pulls their weight a little more than the other. But on the whole, we’re saving hours of time every day already, and look forward to the balance shifting even further to ARCHI’s side.

It’s also worth mentioning that Chord Charts themselves continue to evolve, and ARCHI will take some time to catch up to that, if it catches up at all. In response to your feedback from the Beta, for example, we’ve recently started experimenting with applying Tones from our library to Chord Charts, instead of using a single default Tone to fit all of them. This is a relatively recent development, and as part of the process we’ll need to go back through our entire Chord Chart library to date, making decisions that are somewhat subjective. It remains to be seen whether ARCHI can help us with this in the long term, but we’ll certainly be looking into it.

“Chord Charts themselves continue to evolve, and ARCHI will take some time to catch up to that.”

In Brief…

There is so, so much I could continue to say about Chord Charts.

A lot of thought and effort has gone into this new feature for Rocksmith+, and into making Chord Charts valuable and fun for all players. And of course, in a more prosaic sense, our Music Analyst teams in Pune and here in Osaka have put a lot of care and attention into a lot of Chord Charts numerically, and into collaborating not just with ARCHI but with each other, too. I’d like to reassure you that none of these Verified Chord Charts will be rushed or done carelessly, despite what other machine learning-based projects may have led you to believe. We’re using ARCHI to analyse and arrange songs the way we’d choose to do it ourselves, at speeds that have not been witnessed since I did my university assignments at the last minute, and with much greater results of course. Importantly, we don’t see Chord Charts as taking away from Rocksmith+’s ability to offer the detailed Authentic Arrangements that past Rocksmith players have known and loved. We still have a worldwide team, larger than ever before, committed to that kind of transcription work. What we’re doing with the Music Analyst team and ARCHI is creating an all-new way to give more playable songs to players than ever before. And despite that, we’ve managed to keep our heads on straight! It’s been a joy working with this team, and I think we would never have gotten this far if it wasn’t full of truly remarkable people.

Hopefully this long, long explanation has given you some notion of what Chord Charts are, and how we’re building ARCHI bit by bit to help us achieve our goal of getting quality Chord Charts to you quickly and without compromising on quality.

I mentioned earlier that Chord Charts are still continuing to evolve, and on that note, I’d like to acknowledge that Chord Charts as a concept seem very guitar-oriented. But rest assured, we haven’t forgotten about bass players. Next time, you’ll hear from more of our developers who will tell you a little more about Chord Charts and how bass players will be able to benefit.

But one last note from me: thanks so much for your time and attention, and I hope you’ll find it’ll be rewarded many times over when you try these Chord Charts in Rocksmith+.

Milo St Clare-Holmes is an Australian composer, music teacher, ex-record label lackey and stage mismanager. After working on indie games in Osaka, he’s excited to bring it all back home working on Rocksmith+.