Chapter 274: A Brand New Architecture

Chapter 274 -209: A Brand New Architecture

Chen Lin then turned to look at Bai Qian. Although Bai Qian didn’t say anything, Chen Lin knew he must be feeling a little anxious.

Chen Lin had promised last week that he would develop a new algorithm within a month. However, Bai Qian had only just met him, and his prior knowledge of Chen Lin was limited to news reports. It was understandable that he lacked confidence.

Chen Lin reassured Bai Qian, saying,

“Dr. Bai, you can start by designing the workflow for implementing the new algorithm and verifying its feasibility.”

Chen Lin paused for a moment before continuing,

“I told you before that I would develop a completely new AI algorithm within a month.”

“When the time comes, you can use my algorithm as a basis to design a new model architecture and then begin training.”

“Shen Yan will also come over from Dr. Pan’s side to help you with it.”

Bai Qian’s eyes lit up when he heard this.

“President Chen, are you saying… you’ve already started researching the new algorithm?”

“That’s right.” Chen Lin nodded. “Starting today, I’ll be dedicating my afternoons to mathematical research.”

“The goal is to find a mathematical expression for a neural network architecture superior to the Transformer.”

Hearing this, Bai Qian grew visibly excited.

“That’s wonderful, President Chen!”

“If you can really develop a new algorithm, then my work will finally have a clear direction!”

Chen Lin smiled but didn’t say much more.

He stood up and patted Bai Qian’s shoulder.

“Keep it up. I have to get back to work, too.”

With that, Chen Lin left the conference room and returned to his desk.

He glanced at the time: 11:00 AM.

There was still an hour until noon.

Chen Lin decided to use the rest of the morning to continue studying some technical details.

He would begin his formal mathematical research in the afternoon.

…

「1:00 PM.」

Chen Lin first took care of some miscellaneous tasks.

He turned on his computer, logged into his email, and glanced at his recent messages.

Most were inquiry emails from potential clients. Sun Yu had already screened them, filtering out the obviously unreliable ones.

The ones that remained were mostly from clients with genuine needs.

Chen Lin noticed an email from Professor Tang Xuebing of the Science and Technology University.

The email mentioned that ever since Chen Lin had helped the joint research team from Yannan and the Science and Technology University resolve the mathematical bottleneck in their solid-state battery research, their work had progressed rapidly. A prototype for the new-generation solid-state battery had already entered the testing phase.

Testing a battery is a rather tedious and lengthy process.

From electrical tests like overcharging, over-discharging, external short-circuiting, and forced discharging, to mechanical tests like crushing, nail penetration, impact, vibration, and dropping… it all required a massive amount of time.

The reason for emailing Yanhai Company this time was that while conducting these tests, the project team was also pursuing other research. A new problem in chemical material science had emerged, one that required Chen Lin’s mathematical prowess to solve.

It requested that if Chen Lin had time, he should reply to the email. The project leads would then contact him via WeChat to schedule an in-person meeting to discuss the details.

Chen Lin thought for a moment and replied to the email. He explained that he was currently engaged in some rather important mathematical research, but he would complete it as soon as possible and contact Professor Tang Xuebing afterward.

After handling his emails, Chen Lin took a look at the company’s financial statements.

Huang Yiyi prepared a detailed financial report and sent it to him every week.

It clearly listed the company’s income, expenditures, and current balance.

Chen Lin scanned it and saw that the company had a balance of just over four million RMB.

This sum was enough to cover the company’s expenses for the next several months.

Besides, Chen Lin himself still had the monthly income from [Little Financial Planner].

So, there was no need to worry about funds at all.

Chen Lin closed the financial report and glanced at the time again.

1:50 PM.

‘It’s about time.’

He took a stack of A4 paper from his drawer, along with a few black gel pens.

Taking a deep breath, Chen Lin closed his eyes and began to organize his thoughts.

His research goal was clear—

design a completely new neural network architecture, one that could retain the Transformer’s parallel computing and global modeling capabilities while drastically reducing its computational complexity.

Specifically, he needed to reduce the Transformer’s O(N2·d) complexity to O(N·d·log N) or even lower.

To achieve this, the self-attention mechanism had to be fundamentally redesigned.

Traditional self-attention calculates the attention weight between every single token in a sequence and all other tokens.

This is what leads to the N2 complexity.

‘So, is it possible to calculate attention for only a subset of tokens instead of all of them?’

‘Or, is there a more efficient way to approximate the effect of self-attention?’

All sorts of ideas began to surface in Chen Lin’s mind.

Linear attention, local attention, sparse attention…

These were all solutions that had already been proposed in the industry.

But they were either not effective enough or too complex to implement.

Chen Lin needed to find a more elegant and efficient method.

At this thought, he opened his eyes, picked up his pen, and began to write and sketch on the paper.

The [Little Mathematician] ability quietly activated.

Chen Lin’s mind seemed to enter a marvelous state.

He could feel ideas emerging in his mind at an unprecedented rate.

Various mathematical tools and theoretical frameworks seemed to be automatically combining, arranging, and optimizing themselves…

Chen Lin’s pen flew across the paper.

Line after line of formulas flowed from its tip.

First, he re-examined the self-attention mechanism from the perspective of the Fourier transform.

The Fourier transform can convert a time-domain signal to the frequency domain, allowing frequency-domain multiplication to replace time-domain convolution.

‘So, could self-attention also be accelerated using a similar method?’

Chen Lin continued his derivation.

He discovered that if the self-attention calculation process was expressed in matrix form and then subjected to eigendecomposition…

‘Hey, this could work!’

Chen Lin’s eyes grew brighter and brighter.

He found that by introducing a special kernel function, the self-attention calculation could be transformed into a linear form.

And this linear form could be calculated recursively, thereby reducing the complexity to O(N·d)!

However, this method came at a cost—

it would lose some of its global modeling capability.

‘What should I do?’

Chen Lin continued to think.

He suddenly had an idea: he could introduce a hierarchical attention mechanism.

Use traditional self-attention within a local range to ensure precision.

Use linear attention on a global scale to reduce complexity.

Then, combine the two through a clever information fusion mechanism…

Chen Lin’s pen moved faster and faster.

The formulas grew increasingly complex.

He was completely immersed in the world of mathematical derivations, having lost all track of time.

Before he knew it, more than two hours had passed.

When Chen Lin finally put down his pen, the A4 paper in front of him was covered in dense formulas.

A good dozen pages of them.

Chen Lin leaned back in his chair and let out a long sigh of relief.

He could feel his mind was a bit groggy and his spirit somewhat fatigued.

However, this exhaustion was nowhere near the level of being completely knocked out.

Chen Lin looked down at the formulas on the dozen or so pages before him, a satisfied smile touching the corners of his mouth.

Although it was still just a preliminary framework, he had already found his direction.