How to Solve Design LeetCode

What is the Design LeetCode Problem?

The Design LeetCode problem asks us to create a coding competition platform with a leaderboard and an execution environment. This problem is very similar to the "Design Online Judge" question and shares many similarities; however, it is more akin to a constantly running competition rather than a one-off. Users should be able to submit code and receive a pass/fail response as well as a score to know how well they performed. This problem is very open-ended and has no one true solution. That being said, this is a common problem asked at top FAANG companies like Amazon, Google, and Meta. Here we'll showcase some key points to discuss if you come across this problem in an interview.

An Example of the Design LeetCode Problem

Design a coding competition platform with a leaderboard and execution environment.

How to Solve the Design LeetCode Problem

To tackle this problem, let’s identify our functional and non-functional requirements first. For this particular question, it’s very important to discern what your interviewer prioritizes in respect to answering this question. It’s also vital to outline the scope of the problem, there is a lot of overlap on the back-end for Design LeetCode and another similar problem: “Design Online Judge.” This distinction is typically important at the L4 level, but less so at the senior level when discussing the solution, as one is a continuous running competition while the other is a one-off. Now let’s look at our requirements:

Functional requirements:

• Users are able to submit and run code.
• Users should be able to run their code against some manually provided inputs.
• Problem server tests whether user code passes.
• Users get some feedback on their code — pass/fail, as well as performance metrics on space/time efficiency.
• User’s code is saved and can be pulled up again where they left off.
• Session handling, users can navigate away from the site and return to their code later.
• Safe code execution (containerize everything — protect yourself from rm -rf / stuff).
• Horizontally scalable user code runners / workers.
• Leaderboard that can inquire about the type of leaderboard.
• Support various programming languages and libraries, including compiled languages.
• Test input files matched against output files, and support custom testing algorithms.

Non-functional Requirements:

• The in-browser IDE, code completion, syntax highlighting, jumping to errors, etc.
• Profile creation.

Extra notes:

• Consider rate limiting and how to best implement it.
• Important to consider failure modes for the message queue and comparison of existing services.
• Consider how to store user submissions.

Main focus

A problem like this has many topics to cover, so candidates often struggle to demonstrate clear separation of concerns. Ideally, a candidate should be able to accurately identify what the main concerns of the system would be and allocate most of their time to these concerns. By doing this, a candidate can transform their proposed solution from a jumbled mess of priorities that is just passable at an L4 into a strong L5 hire.

With that said, the two pillars of this problem are sandboxing user code execution and the task queue to efficiently scale execution across multiple workers.

Sandboxing:

To ensure safe code execution, sandboxing is a critical component. By using containerization technology like Docker, isolated execution environments can be created for running user code. Each user's code runs within its own container, separate from the host system, providing a sandboxed environment. Techniques such as resource limitations, network isolation, file system restrictions, timeouts, and execution limits help enforce security within the containers.

Storage:

A reliable storage system, such as a SQL database, plays a crucial role in the coding competition platform. It enables users to revisit questions and easily access their code submissions. The storage system supports the leaderboard functionality by storing user rankings, performance metrics, and historical records, allowing the leaderboard to accurately reflect user standings.

Session IDs:

Incorporating session IDs for virtual machines can improve our system. Session IDs can be assigned to each user's virtual machine instance, allowing individual sessions to be tracked and managed.

Static Content:

Your interviewer might ask you about how you would retrieve and display static content such as problem statements. You can mention CDNs, S3 CloudFront, and spend a few minutes explaining how it works and note that they are cheap and low latency.

Caching:

When it comes to caching content, static content would be loaded first, including problem statements, test cases, and images.

Message Queue (Kafka)

To avoid putting a heavy load on our system, we’ll need a message queue to handle our user submission file processing asynchronously. One possible option is Kafka, a distributed streaming platform known for its high throughput and fault-tolerant design. It is suitable for handling large volumes of problem grading requests.

Leaderboard:

For the leaderboard presentation, sortable tables can be used to display the leaderboard, allowing users to sort by criteria like runtime, space complexity, or overall performance.

Scalability/Reliability:

Leveraging cloud infrastructure services from major providers like AWS ensures high availability and fault tolerance. Implementing automated backup and disaster recovery mechanisms protects against data loss and enables quick recovery.

Non-functional:

The platform should provide features that enhance the coding experience, such as syntax highlighting and module import.

Senior-level capacity planning:

Judging user-submitted code is very CPU-intensive, and the bill can easily run 1000’s or 10000’s of dollars in AWS EC2s or Lambdas. This could save the company hundreds of thousands of dollars per year.