True Colors: Research team adjusts how TVs read team colors

September 28, 2020

A view of how Clemson University’s football team’s purple jerseys appeared on televisions around the country.

The Saturday before Thanksgiving in 2018, my family and I settle into our living room to watch the Clemson Tigers play the Duke Blue Devils. I was lucky to be able to fly home a little before the break began and was thrilled to be able to watch the football game with my family. My mom prepared for the occasion, filling the coffee table with snacks usually reserved for our Super Bowl watch party. As the camera pans to the Tigers making their run down the Hill as part of the most exciting 25 seconds in college football, my brother only makes one comment: “Oh, Duke is running down the Hill!”

Emma Mayes, a Clemson student on Dr. Walker’s research team

Of course, it wasn’t Duke. There is no way an opponent would run down the Hill at a Clemson home game. But lo and behold, it appeared that way. Clemson’s purple jerseys donned to celebrate Clemson’s “Purple Out” football game in honor of its Military Appreciation Day, appeared navy blue on the screen, a similar shade to Duke’s team colors. And it wasn’t just our TV; Clemson looked to be wearing navy blue across broadcast footage and television screens. My dad and brother had a field day, cracking jokes about how Duke was playing themselves in Death Valley.

This is not a problem limited to Clemson. This brand color discrepancy is also apparent with other teams when they play, such as the Dallas Cowboys, where their silver pants appeared green during a broadcasted game. And what is a sports team without their team colors? Unfortunately, trying to correct the brand colors means all the colors in the frame would suffer. Environmental lighting and post-production color adjustments affect the entire frame of the broadcast feed. So, while the brand color could be adjusted to appear correct, the rest of the frame would be adjusted as well, ruining other colors at the expense of fixing one.

Dr. Erica Walker, a Clemson University graphic communications professor

Enter Dr. Erica Walker, a Graphic Communications professor at Clemson University, who brought together a team that asked whether the frame could be adjusted so these colors appear to brand specification, but not at the cost of the rest of the frame being adjusted as well. She partnered with the Watson in the Watt Creative Inquiry run by Dr. Hudson Smith and Dr. Carl Ehrett to work with undergraduate students to develop an AI-based solution. I ended up being one of those undergraduate students. I was thrilled to get involved with research about AI but was even more excited by this problem we would be solving, remembering that this was something my own family had commented on weeks before.

Once a week, as the ColorNet research team settles into their meeting room in the Watt Family Innovation Center and exchanges what their week looks like, Dr. Walker comes into the team room with a big smile on her face. “Hey y’all, how’s it going?” she says as she sets down her bag. It always starts the weekly meetings on a good note and the team can’t help but be excited about what they will accomplish that day.

And accomplish great things we did. With the efforts of Dr. Walker, ColorNet was developed, a patent-pending artificial intelligence (AI) technology that takes a live video feed and adjusts it frame-by-frame, pixel-by-pixel, in real-time to ensure brand color accuracy. This convolutional neural network, a type of machine learning model, was trained on numerous clips from Clemson’s football games that were manually color corrected using Adobe Premiere Pro. ColorNet learns what areas of the frame are considered brand colors and how to shift those pixels so that they appear to visually match Clemson’s brand colors without losing the distinctions between highlights and shadows and without changing the non-brand color areas.

Michelle Mayer, a Clemson student on Dr. Walker’s research team

After training the model to color correct Clemson’s orange, Pantone 165, it was time to test it. All testing took place in Littlejohn Coliseum, allowing the technology to be tested on a standard jumbotron and within the current infrastructure of a live event. The alpha test involved streaming one of Clemson’s home football games into Littlejohn and onto the jumbotron to determine if ColorNet would catch the various spots of Clemson orange in each frame and adjust these areas to be the recognizable color fans know and love. With some adjustments to the program and support provided via pizza from Dr. Walker, the alpha test proved to be a success.

The beta test took place during a live women’s basketball game, where we ran ColorNet on the jumbotron feed during the basketball game in front of a live audience of fans. The goal was to determine if ColorNet was truly targeting areas of brand color and was able to be extended to other sports or broadcast arenas. The technology proved itself once again! It correctly picked out the women’s jerseys and the splashes of Clemson orange found on the spectators in the arena. I remember walking the arena with Dr. Walker, both of us looking up at the jumbotron in wonder at what we had accomplished with ColorNet.

Our work is by no means over. The goal is to make it where we can color correct any team’s colors, not just Clemson’s. While we were able to expand ColorNet’s capabilities so it can also color correct Clemson purple along with orange, the goal is to generalize the application of the model so that it can color correct to a given color based on user input. Are we playing Alabama? Let’s correct for Crimson Tide red as well. LSU? Let’s adjust for their purple and gold. While COVID-19 makes it hard to meet and collaborate in person towards this development, the meetings are still opened the same. And the same excitement for where ColorNet will go next is tangible even through the screen, thanks to the passion of Dr. Walker.