From Finger Gymnastics To Chain-of-Thought

Vim to Cursor

The creator of Vim, Bram Moolenaar, passed away on 3rd of August, 2023. In the same year, Cursor, one of the most popular AI IDEs today, raised $20 million in Series A funding, after it had been founded for one year. Before the AI-assisted development tool arose, programmers had “respected” those who are able to code fast with their fingers, and considered it as a sign of coding prowess. When I firstly saw my senior programmer typing code without leaving his keyboard, I was amazed. It was so cool that he typed codes just like the hackers in the movies.

Today, the coolest way of programming seems no longer to be typing codes with hotkeys completely on the keyboard, but to be merely clicking “tab” and “enter” to generate codes with AI - tab to autocomplete and enter to generate/accept. This is thanks to the proliferation of the large language models (LLMs) that are used for coding efficiency improvement.

After my amazement, I started to wonder: what are the implications of this new programming paradigm? What are the pros and cons?

What do we gain and what do we lose?

The gain is very observable

This is what I usually do when I am working on coding activities as a machine learning engineer without AI.

1. Get a request of application or a feature from the client or the product manager.
2. Think about what I want to do, and how to do it.
3. Open my IDE (without a loss of my heavy preference for Vim, I use VSCode with Vim emulation plugin).
4. Draw some sketches about what the code should look like - usually I do this with mermaid or draw.io.
5. Write down the thoughts into the markdown files.
6. Design and develop the structure of the code. Not actual code yet, but just the scaffolding. In Python, I use poetry for this.
7. Write the high-level interface and base classes.
8. Write the actual functions and classes. I am a data scientist but I don’t write in Jupyter notebook. My codes directly run as scripts with unit tests (Test-Driven Development, TDD).
9. Develop/refine the CI/CD pipeline. I use Github Actions or Azure DevOps.
10. Deploy the application or feature. I use Kubernetes for this.

To accelerate the development process above, I have my own code base where there are many components that can be reused. For example, I have a code base for data processing, a code base for model development, a code base for deployment, and so on. I put efforts into refining these code bases so that I can save more time for the development of the new feature. But still, for any brand-new applications, I have to write lots of codes manually, particular, the tests.

With the AI-assisted coding…

I can probably just need to do the first three steps above! This leads to a significant gain of efficiency in the development process. Moreover, I am not a full-stack programmer, so I can just focus on the interface and the core of the application. I don’t need to care about the programming language, the framework, nor the low-level details!

Well, the gain is not just for me. It is for nearly everyone that wants to develop software without a thorough understanding of the low-level coding details. And amazingly, we saw the fantastic work from even kids who used AI to generate codes for their projects (see below). With the adoption of the AI-assisted coding, the number of potential programmers (e.g., non-computer science, non-engineering students) has increased dramatically. And truly building software application has been democratized. And it’s no longer the privilege of the computer science students or the engineering professionals any more!

But is it all good?

Emotional fulfillment as a programmer may not be there

After my initial excitement, one significant aspect that may be lost in the transition to AI-assisted coding is the emotional fulfillment. I became “lazy” to work on a Vim-like environment any more after I found that an AI IDE like Cursor can generate codes for me with just typing “tab” and “enter”. The AI-based tool also diminishes my “pride” as a Vim programmer. Indeed, I can do things faster with auto-completion but, that is not the coolness of me - it is that of the AI models. I may not be respected any more as a programmer that can code fast with Vim - I am simply downgraded to be equivalent to an AI model that is tuned to just generate codes for me.

The picture below shows the IDE experience with Vim and its plugin, called nerdtree. See reference
here (it was created by @jistr 10 years ago!).

To achieve the above, assuming the plugin of nerdtree is installed, you can use the following steps:

Place this in your .vimrc:

Plugin 'xuyuanp/git-nerdtree'

then run the following in Vim:

:source %
:PluginInstall

Now this may be treated as a quite “manual” and “tedious” way of coding because AI can do everything for us. But the fulfillment of joy from configuring a Vim IDE lasts much longer than that from developing an application with Cursor.

While my fulfillment is from the use of the IDE for coding my own application, other programmers may have their own fulfillment from other things. For example, some may find fulfillment from finding a more beautiful pattern to design a data access layer, implementing a new algorithm with lower time complexity compared to the state-of-the-art, or optimizing the data pipeline efficiency by introducing proper partition in the database tables. All of these joys, with the use of AI-based tool, may be gone, because they don’t need to and perhaps, don’t want to think about these things.

And this leads to the next point - AI programming tools may gradually erode programmers’ pursuit of technical depth and details without them realizing it.

No more need for in-depth understanding about the low-level details

Think about the last time you were debugging a C or C++ code when the frustrating “core dump” error frequently occurs in the console and you had to dig into the code to find the root cause day and night. And the debugging process requires you to not just fix the bug with some “trials-and-errors” but to really understand the low-level details of the memory management and system architecture. A similar situation presents for the coding exercise of a data scientist. In the data science project, the AI-assistant can easily produce the codes for data transformation, feature engineering, etc., but the data scientists or data engineers will lose the opportunities to have an in-depth understanding of the best practices in handling data with regard to the data I/O performance, analytics performance, and other aspects.

The AI IDE also creates a significant impact on the technical interview process for programmers. Leetcode or Hacker Rank are the most popular platforms for programmers coding interviews. Though I am not a believer in the effectiveness of the Leetcode-like platform for evaluating the programmer’s skills, it is undeniable that it provides programmers with the opportunities to practice and sharpen their skills on the low-level details of the programming. It is very likely that the assessment of Leetcode questions are never practiced in the actual work. However, the programmers who are able to solve the problems correctly in the coding interviews are more likely to be the ones who are able to perform well in the actual work. And the good practices that have been developed in the Leetcode question preparation will be beneficial to the programmers in the actual work as well. NOTE the purpose of practicing Leetcode is definitely not to memorize the solutions to the problems like a machine - very apparently AI does better than humans in this aspect. The Leetcode practice is to train the programmer’s skills to think about the problems deeply and to be creative with the solutions. After being used to the AI-based coding, the programmers may not be able to think creatively and deeply like before.

Is coding a collaborative activity any more?

Like everyone else, I got impressed by the YouTube video above where a 8yo kid codes a simple application by herself with AI. There are tons of similar videos, blogs, and GitHub repositories where the creators want to demonstrate one thing - AI can help programmer to easily handle the development of work solo projects. My question then is: does it mean that collaboration on the code is no more needed, and a pair of human + AI is better than a pair of humans?

There are lots of rebuttal against the capability of AI to do large-scale collaborative projects. For example, the AI may not be able to understand the context and the humor in the code, or the AI may not be able to collaborate with the programmer to solve the problem. I accept that “composer” in Cursor does a great job in understanding not just the code but also the overall project context. This works for a solo project that builds, say, an application that helps me manage my personal finance, arrange my travel with family, etc. But what about the cases that the project is complex and requires multiple human programmers to collaborate with each other? Like a recommender system that is used by a ecommerce company? I doubt the capability of the contemporary AI-based tool is sufficient to a single program for the entire development.

Also, AI breaks the conventional workflow of the collaborative coding work that human programmers have been following for decades. git (definitely GitHub, too) is one of the most popular tools for collaborative coding, and it is not only useful to help version control but also help collaboration among a group of human programmers. There are tools and practices for version control of the prompts that are used for generating codes, too (e.g., Dify, langchain, Pezzo, etc., are examples of the LLMOps tools that help managee and version control the prompts). Nevertheless, due to the nature of the LLMs, it is hard to reproduce exactly the same codes with the same prompt, and the collaboration among the programmers may not be as straightforward as before.

Responsibility and accountability?

The AI-based approach is different from the traditional approach in terms of who is responsible for the code. In the traditional approach, the programmer is responsible for the code from the prompt to the solution. But with the AI-based approach, the programmer is responsible for the prompt, and the AI is responsible for the code - what if the code has issues?

Again, this is the classic issue of any predictive model - for any model that is probabilistic (apparently, this is definite for all of the machine learning models where patterns are learnt from samples), there is no guarantee that the model will produce the same output for the same input, and thus, the operationalization of the model should be overseen by a human for accountability and responsibility.

One may say, the programmer is accountable for the codes because the the codes are generated from the prompts which are provided by the programmer. But this seems not sensible, because the programmer may say, “The application actually runs on the codes generated from the LLM, not my prompts”. And the programmer may also say, “My prompts are not responsible for the codes because no one can reproduce exactly the same codes with the same prompt!” In fact, these are valid points. This is indeed neither a problem of the programmer nor the AI. It’s about the protocol with which the model should be used with sufficient ethical concerns. There are quite some good thoughts on this topic from the industry and academia (see references below).

What shall we do?

I believe that the AI-based programming tools will further evolve and become more sophisticated. There will be more advanced tools that can help programmers to be more productive. And there will be more advanced tools that can help programmers to be more creative. I don’t have a anticipation on what it will be like.But I will still have my Vim plugin installed in both VSCode (it is where I use GitHub Copilot) and Cursor. I am so used to the Vim hotkeys and perhaps for my whole life I will use it. My Vim-based programming habits coexist with AI-based programming assistance tools, ensuring that I retain a bit of the traditional programmer’s dignity. But I don’t know if in a very near future, whether there will be another significant shift of the paradigm again about how humans “program” (well can we still call it programming when it is done by AI? 😅).

Hope I am not too pessimistic.

References

• Vim
• Cursor
• Copilot
• Building Trustworthy AI Systems
• AI-generated code demands ‘trust but verify’ ethic
• AI-Generated Code is Causing Outages and Security Issues in Businesses,
• Trust and accountability in AI-generated code

Citation

Plain citation as

Zhang, Le. From Finger Gymnastics To Chain-of-Thoughts. Thinkloud. https://yueguoguo.github.io/posts/2024/09/07/ai-tool/

or Bibliography-like citation

@article{yueguoguo2024cot,
  title   = "From Finger Gymnastics To Chain-of-Thoughts",
  author  = "Zhang, Le",
  journal = "yueguoguo.github.io",
  year    = "2024",
  month   = "Sep",
  url     = "https://yueguoguo.github.io/2024/09/07/ai-tool/"
}