Chapter 273 - 208: Renting a Server

Chapter 273: Chapter 208: Renting a Server

November 25th, a Tuesday.

After being processed and disseminated by major media outlets, the content of Chen Lin’s interview from yesterday quickly blew up online.

Weibo, Zhihu, Douyin...

News about Chen Lin could be seen on nearly every social media platform.

And the most attention-grabbing piece of news was the information Chen Lin had revealed—

Yanhai Company was going to make a large AI model!

As soon as this news broke, it immediately caused a huge sensation in the tech world.

On Zhihu, someone created a dedicated question thread:

[What are your thoughts on Chen Lin revealing that Yanhai Company will enter the field of large AI model training?]

In just one day, over a thousand answers flooded the thread.

Some were optimistic:

[Chen Lin’s strength in algorithms is plain for all to see. If he’s personally getting involved in training a large model, I think the probability of success is very high.]

[Yanhai might be a small company, but you can’t deny how badass Chen Lin is! Other companies compete on funding and computing power, but Chen Lin can just deliver a dimensional-reduction strike from the algorithmic level!]

Others were skeptical:

[Training a large model is no joke. Just data cleaning and annotation alone require massive amounts of manpower and capital. Can a small company like Yanhai really handle it?]

[I think Chen Lin is getting a little arrogant. He’s definitely strong in mathematics, but creating a large AI model isn’t just a math problem. It also involves a whole series of issues like engineering implementation, computing power scheduling, data processing, and so on. These aren’t things a single math genius can solve.]

And some were just joking around:

[Chen Lin: I’ll just casually whip something up. CloseAI: ???]

[Ultraman and Jask will probably lose sleep over this news, hahaha!]

[Can’t wait for Chen Lin to raise the bar so high it kills the rest of the AI industry!]

The online discussions were raging, but Chen Lin himself was no longer paying attention to them.

At this moment, he was sitting in the Yanhai Company office, staring at his computer screen with rapt attention.

The screen was densely packed with technical documents about neural networks and the Transformer architecture.

Chen Lin had already spent the better part of half a year teaching himself about large AI models.

And now, he was pleasantly surprised to discover that this knowledge could actually be combined with his [Little Mathematician] ability!

Previously, when Chen Lin was solving client problems, he often needed people like Shen Yan or Huang Yiyi, who studied applied mathematics, to first "translate" the engineering problems into pure mathematical problems before he could use his [Little Mathematician] ability to solve them.

But now, in the field of artificial neural networks, Chen Lin found that he no longer needed a "translator."

He could directly understand the structure and principles of these neural networks and could directly contemplate how to optimize their structures from a mathematical perspective.

And as soon as he started thinking, his [Little Mathematician] ability would automatically activate, generating the complete solution process in his mind.

’This feels absolutely incredible!’

The more Chen Lin thought about it, the more excited he became, so much so that he wished he could start his research immediately.

However, he still forced himself to calm down and finish reading the remaining technical documents.

The current mainstream Transformer architecture is a feature extraction network structure based on a self-attention mechanism, primarily used in the field of natural language processing.

The self-attention mechanism can perform attention calculations for each position in the input sequence, thereby acquiring global contextual information.

The core of the Transformer is the self-attention mechanism.

It can process entire sentences of natural language at once, leading to higher training efficiency and enabling parallel processing.

But the Transformer also has obvious drawbacks—

It has high computational complexity, with a time complexity of O(N2·d), where N is the sequence length and d is the dimension of the token embedding.

This time complexity is very unfriendly to long sequences.

For example, to process an article of 10,000 characters, the number of calculations would be 10,000 squared, which is 100 million calculations.

What about processing a long text of 100,000 characters?

That would be ten billion calculations.

The consumption of computing power for this is simply astronomical.

Therefore, the industry has been constantly searching for methods to reduce the Transformer’s computational complexity.

Chen Lin’s research goal was to combine the advantages of linear models and attention mechanisms, allowing the model to perform parallel computations and improve computational complexity while achieving results comparable to the Transformer.

This goal sounded simple, but its implementation was extremely difficult.

Because it was equivalent to redesigning an entirely new neural network architecture.

However, Chen Lin was confident.

Because he had [Little Mathematician].

As long as he could think the problem through clearly and set a definite goal, the remaining mathematical derivations would not be an issue.

Thinking of this, Chen Lin took a deep breath and adjusted his work schedule.

Starting today, he would continue to study knowledge related to large AI models every morning, and also discuss the work progress with Pan Si and Bai Qian.

In the afternoon, he would conduct two hours and forty minutes of mathematical research.

This schedule was the same as when he was preparing for his speech at the mathematics conference.

Experience had shown that this rhythm was the most suitable for Chen Lin.

It not only ensured the efficiency of his learning and research but also prevented the overuse of his [Little Mathematician] ability, which could cause him to fall into a dead sleep.

Chen Lin glanced at the time: ten in the morning.

Pan Si and Bai Qian should already be at the company.

He stood up and walked over to Pan Si and Bai Qian’s desks.

Pan Si was sitting in front of his computer, staring at the dense code on the screen.

Bai Qian was sitting on the other side, a pile of scratch paper covered in mathematical formulas spread out before him.

"President Chen." Pan Si looked up and nodded at Chen Lin.

Bai Qian also stopped writing and pushed up his glasses. "President Chen, what’s up?"

"I just came over to check on your progress." Chen Lin pulled up a chair and sat down. "Dr. Pan, how are things on your end?"

Hearing this, a trace of excitement appeared on Pan Si’s face.

"President Chen, my progress has been pretty good!"

He turned his computer screen so Chen Lin could see its contents.

"I’ve already set up the data cleaning pipeline, and now Shen Yan and Sun Yu are helping with the data annotation."

"Right now, we’ve cleaned about twenty million high-quality Chinese and English corpus entries."

"At this rate, in another two weeks, we should be able to accumulate around fifty million entries."

"By then, we can start the first round of pre-training."

Chen Lin nodded with satisfaction upon hearing this.

"Excellent. What about the servers? Have you figured out a plan?"

"We have." Pan Si said with a smile, "Dr. Bai and I discussed it and decided to go with a plan to rent cloud servers."

"In the initial stages of the project, we’ll be training and validating models with a small number of parameters, keeping the model parameters under 30B."

"In that case, renting cloud servers will be sufficient. The monthly cost will be around 100,000 US dollars."

"Although the price isn’t cheap, it’s actually quite cost-effective because Dr. Bai and I can share computing resources."

Chen Lin nodded.

100,000 US dollars a month is about 700,000 RMB.

For Yanhai Company’s current financial situation, this was completely affordable.

"Then let’s proceed with this plan," Chen Lin said. "Tally up the required expenses and report them to Assistant Huang. She’ll arrange the payment."

"Understood, President Chen," Pan Si replied.