/Private / md-notes

System Design

2026-09-02T05:29:25.021Z

Introduction

What is System Design?

System design is the blue print of any software system, it is a the strategic process of defining the architecture, components, modules, interfaces, and data flow for a software system to satisfy specified business and technical requirements.

1. Scalability

Often you hear in a discussion “but that doesn’t scale” as the magical word to end an argument. This is often an indication that developers are running into situations where the architecture of their system limits their ability to grow their service. What is it that we really mean by scalability?

A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added. Increasing performance in general means serving more units of work, but it can also be to handle larger units of work, such as when datasets grow.
When it comes to distributes systems adding resources can have many more reasons like to improve the reliability of the offered service and many more.

→ BUT WHY IS SCALABILITY HARD !? Because…

It requires applications and platforms to be designed with scaling in mind, such that adding resources actually results in improving the performance or that if redundancy is introduced the system performance is not adversely affected.
Many algorithms that perform reasonably well under low load and small datasets can explode in cost if either requests rates increase, the dataset grows or the number of nodes in the distributed system increases.

1.1 Types of Scaling

Horizontal and vertical scaling

Horizontal scaling and Vertical scaling both involve adding resources to your computing infrastructure, you must decide which is right for your application. Scaling horizontally and scaling vertically are similar in that they both involve adding computing resources to your infrastructure. There are distinct differences between the two in terms of implementation and performance.

Horizontal scaling means scaling by adding more machines to your pool of resources (also described as “scaling out”), whereas Vertical scaling refers to scaling by adding more power (e.g. CPU, RAM) to an existing machine (also described as “scaling up”)
One of the fundamental differences between the two is that horizontal scaling requires breaking a sequential piece of logic into smaller pieces so that they can be executed in parallel across multiple machines.
In many respects, vertical scaling is easier because the logic really doesn’t need to change. Rather, you’re just running the same code on higher-spec machines.

However, there are many other factors to consider when determining the appropriate approach. So lets explain one by one.

1.1.1 Vertical Scaling

Vertical Scaling basically makes one node stronger, if you have 1 server, make the server stronger with adding more hardware.
Vertical scaling keeps your existing infrastructure but adding more computing power. Your existing code doesn’t need to change — you simply need to run the same code on machines with better specs. Vertical scaling means adding more resources to a single node and adding additional CPU, RAM, and DISK to cope with an increasing workload.
It’s important to keep in mind that you can only increase it to the limits of your server. By scaling up, you increase the capacity of a single machine.
Vertical scaling allows data to live on a single node, and scaling spreads the load through CPU and RAM resources for your machines. But the system has hardware limitations that you can increate CPU and RAM. Also this will highly expensive when you reach the maximum capacity.
If you have get millions of request, in that case having one server won’t be enough, because even the hardware has maximum capacity limitations.

1.1.2 Horizontal Scaling

Horizontal scaling means adding more machines to the resource pool,
rather than simply adding resources by scaling vertically. Vertical scaling gives you the ability to zoom into add more servers to your network, but it also requires you to zoom out by adding a bit more power, CPU, and RAM to the existing infrastructure.
Scaling horizontally gives you scalability but also reliability because you will have more redundancy and mostly its the preferred way to scale in distributed architectures.
If your services are stateless, then its easy and the best practice to Scaling horizontally.
We can just put the different services and have a load balancer that will split the load traffic to different servers.

2. Caching

Caching, in the world of computer systems, is a technique to store a copy of data or computational results that can be retrieved quickly. The cache memory is like a tiny treasure box that keeps the most frequently accessed data ready for you.

Caching adds speed and reduces latency, and lightens the load on the Backend by providing the prep-rocessed data.

2.1 Caching: Mechanisms, Tools, and Techniques

Browser Caching - Browser caching stores webpage resources on a local computer when a user visits a webpage. On the next visit, the browser loads the page from local resources, reducing server load and improving page load speed.Browser caching is powerful but comes with limitations it’s only effective for repeat visits and is entirely client-dependent.
CDN Caching (Content Delivery Network) - CDN caching replicates your data across various geographical locations, reducing the data travel time (latency) by serving it from the nearest server to the user. Big players like Cloudflare and AWS CloudFront provide CDN services. CDN caching shines when you have a global user base. The more distributed the users, the more beneficial a CDN becomes.
Database Query Caching - Database query caching saves the result set of a query in the cache. When the same query is executed, the DBMS first checks the cache. If found, the results are returned from the cache, skipping the actual database hit.
In-Memory Caching (e.g., Redis, Memcached) - In-memory caches store data in RAM, which is much faster than typical disk storage. They are often used for frequently read data, session information, and full-page caching.
Application Caching - Custom build application cache.
Distributed Caching - Application cache, but in microservice architecture.

2.2 Cache Strategies: Types and When to Use What

A well implemented cache can improve the performance of a system but the only deal to make it work efficiently is to choose the correct caching strategy.

Cache-Aside(Lazy Loading) - In the Cache-Aside strategy, the application looks for data in the cache first. If it doesn’t find the data (a cache miss), it loads data from the source, stores it in the cache, and then returns it. This strategy ensures that only requested data is cached, thus preventing the caching of unnecessary data. This strategy is best when reads are much more frequent than writes, and data doesn’t get updated often.
Read-Through (Eager Loading) - In the Read-Through strategy, cache is completely abstracted from the application. When a cache miss occurs, the cache itself is responsible for reading data from the data source, storing it, and then returning it. This approach maintains data consistency and saves the application from interacting with the data source. This strategy works best when the cost of a cache miss is high and when you want to keep your application logic simple.
Write-Through - In the Write-Through strategy, every write operation to the database is simultaneously written to the cache. This ensures the cache always holds the latest data, reducing the chance of serving stale data. This strategy is suitable when write operations are less frequent and you need up-to-date data in cache at all times.
Write-Back (Write-Behind) - In the Write-Back strategy, write operations are first written to the cache, and the cache then writes the data back to the database after a certain delay. This strategy significantly reduces database load. This strategy is ideal when write performance is more critical than data consistency, and the risk of data loss is acceptable.
Refresh-Ahead - In the Refresh-Ahead strategy, cache proactively refreshes data before it gets stale or expires. This strategy ensures that data is always ready in the cache, reducing cache miss latency. This strategy is great when you have predictable data access patterns and can afford the overhead of pre-fetching data.

Choosing the Right Caching Strategy

Now that we know the options, how do we choose the right caching strategy? Several factors come into play:

Scale of the Application: Larger applications with heavy traffic need more robust caching solutions like CDN or distributed caching.
Type of Data: Highly dynamic data might not be the best candidate for caching. On the other hand, data that is expensive to compute or doesn’t change often should be prioritized.
User Distribution: If your users are globally distributed, consider CDN caching. If most of your user base is local, in-memory caching may be more effective.
Infrastructure: Cloud-based applications can take advantage of the built-in caching mechanisms provided by cloud vendors.

3. Load Balancing

In the world of computers and the internet, when a website or online service becomes super popular, it starts receiving a flood of requests from users all over the place. Instead of relying on a single server to handle all these requests, load balancing spreads the workload across multiple servers.

No single server gets overwhelmed with too much traffic.
Every server gets to share the load.
Prevents crashes, slowdowns, and annoying error messages, making sure that users have a smooth experience while browsing websites or using online applications.
Load balancing algorithms decide which server should handle each request, making sure they’re divvied up fairly.

3.1 Types of Load Balancers based on the OSI Model

Network Load Balancer (Load Balancer 4):
- Operates at the Transport Layer (layer 4) of the OSI model.
- It makes decisions based on information such as the source and destination IP addresses, as well as the port numbers of incoming requests.
- The main focus of Layer 4 load balancers is to distribute network traffic efficiently across multiple servers.
- When a user sends a request to access a website or application, the Layer 4 Load Balancer receives the request. It then looks at the transport layer data (IP addresses and ports) to determine which server should handle the request. The load balancer uses various algorithms (e.g., round-robin, least connections) to decide the best server to forward the request to. This ensures that traffic is evenly spread among the servers, improving performance and avoiding overloading any single server.
- Cannot make application specific desicions
Application Load Balancer (Load Balancer 7):
- Operates at the application layer.
- It can make more intelligent decisions based on application-specific data, such as HTTP headers, cookies, and URLs.
- Load Balancers at this level understand application protocols, enabling them to optimize traffic distribution for specific applications or services.
- When a user sends a request, the Layer 7 Load Balancer analyzes the content of the request to gain insights into the application being accessed. For example, it can identify the type of service (e.g., HTTP, HTTPS, FTP) or the specific URL being requested. Using this information, the load balancer can make intelligent decisions about which server is best suited to handle the request. This allows for more advanced load balancing strategies, such as sending certain requests to specialized servers that can handle specific application tasks.
- Higher processing overhead compared to Layer 4 Load Balancer due to content analysis.

3.2 Load Balancing Algorithms

3.2.1 Round Robin:

Transport Layer algorithm.
It maintains a pool of servers for handling incoming requests.
When the request comes the load balancer starts with the first server and sends it the request and then continues sending the requests in the linear or the circular manner.
In a long run each server receives the equal number of requests.
Simple, no sessions required, no external dependency.
No health monitoring, unequal server capability, no geo-location awareness.

3.2.2 Weighted Round Robin:

Expansion of basic round robin algorithm.
It fixes the issue of unequal server capacities by assigning different weights to each server in the pool, the weights represent the relative compute and performance capability of the servers.
When a new request comes in, the load balancer uses the weights to determine which server should handle the request. It cycles through the servers in a circular manner, as in the basic Round Robin algorithm, but with a slight modification. Instead of equal distribution, it takes into account the weights of the servers.
Fine tuned balance, capacity base distribution of requests.
Complex, static config.

3.2.3 IP Hash

Application layer algorithm.
Distribute incoming network traffic across multiple servers based on the source IP address of the client.
From the incoming request the load balancer extracts the ip address of the request.
The IP address is then hashed, meaning it is converted into a unique numerical value using a hash function. The hash function ensures that the same IP address will always produce the same hash value.
The load balancer uses the hashed value to map the client’s IP address to a specific server in the pool. This mapping is typically done by dividing the hash value by the total number of servers and selecting the corresponding server.

3.2.4 Least Connection Algorithm

Based on number of active connection of a server.
The load balancer continuously monitors the number of active connections of each server in the server pool.
When a new request comes in, the load balancer evaluates the current connection count for each server and selects the server with the fewest active connections.

3.2.5 Least Response Time Algorithm

Based on the response time of the server on the previous requests.
The load balancer continuously measures the response times of each server in the pool to previous requests.
When a new request comes in, the load balancer evaluates the response times of each server and selects the server with the shortest response time.

Time to First Byte (TTFB) is the time taken by a server to send the first byte of data in response to a client’s request. It includes the time spent on processing the request, database queries, and any other backend operations before the server starts sending the response data. The Least Response Time load balancing algorithm implicitly takes into account the Time to First Byte as it evaluates the overall response time of each server. The response time includes the TTFB along with the time taken to transmit the complete response data.

3. Database Replication and Partitioning

3.1 Database Replication

Before getting started you must know some keywords for better understanding.

Latency - It is the total time a system takes to process a request and return a response.
Availability - It is a measure of percentage of time a system is operational and accessible to users.
Fault tolerance - It is a system’s ability to continue operating properly without interruption when a hardware, software, or network component fails.
Data Throughput - It is the total volume of data or number of operations a system can process, transfer, or handle successfully within a specific time unit.
Data Consistency - Ensures that all nodes in a distributed system see the same data at the same time, preventing conflicts during concurrent access.
Durability - Ensures that once data is committed, it remains intact, permanent, and recoverable, even during system crashes, power failures, or hardware outages.

What is Database Replication? Database replication in simple language means to store data at one or more places or simply copying the data from one database or from one server to another.

BUT WHY DO WE NEED IT???? Let us take an example situation to understand the need in a better way. I am running an E-comm company and my company is a huge success in India but now i am targeting multi national costumers, so if a person from Australia comes to visit my website he or she will be able to see the products after a large amount of time which is a problem as my servers are hosted in the Indian region. Increasing the resources on my server is not the solution of this problem. The issue of because of network latency.So by replication, we can keep data close to the user’s geographic area to reduce latency. Similar things are done by Netflix and Amazon prime video etc that’s why you people can watch HD movies without any buffering delays.

Replication also solves the problems like Availability & Fault toleration issues and Data throughput issues. So as a CEO of an e-commerce startup, if my application goes down (Unavailable) then my whole business could get impacted and I could have lost thousands of bucks and can lose the trust of our customers which is very bad these days. Hence we need better Fault tolerance in our system.

3.2 Algorithms to implement Database replication

Single Leader Replication(Active-Passive or Master-Slave Replication)
Multi-Leader Replication
Leaderless Replication

System Design MOC

2026-09-02T05:29:25.021Z

🗺️ System Design Map of Content

Welcome to the syllabus and index for System Design.

📄 Notes

System Design

Md Notes MOC

2026-09-02T05:29:25.021Z

🗺️ Md Notes Map of Content

Welcome to the syllabus and index for Md Notes.

📁 Sub-Topics

📄 Notes

Index
README

key

2026-09-02T05:29:25.021Z

bases renderer of Private/md-notes/key

Index for Devops

2026-09-02T05:29:25.020Z

OWASP, GitOps,Infrastructure Monitoring,Kubernetes,Docker,Networking for Devops,Nginx,OWASP,AWS,CI CD,Ansible,,Terraform,Consul,git-cheat-sheet.pdf,Service Mesh

Kubernetes

2026-09-02T05:29:25.020Z

*prerequisite - Docker

Kubernetes (often abbreviated as K8s) is an open-source platform that automates deploying, scaling, and managing containerized applications.

→ Kubernetes is a Cloud Native Computing Foundation (CNCF) graduates project which is responsible for orchestrating containers.

Note

Monolithic architecture is a single, unified unit where all components are tightly coupled, making it easier to develop initially but harder to scale. Microservices break applications into independent, loosely coupled services, allowing for independent deployment and better scalability, but with higher complexity

1. Kubernetes Architecture

Master Node = Headquater Worker Node = Branch(runs docker containers)

etcd - Consistent and highly-available key value store for all API server data.
Controller Manager - Runs controllers to implement Kubernetes API behavior. [Project Manager] Supervises that everything is working fine of not. Works at cluster level
Api server - The core component server that exposes the Kubernetes HTTP API. (communication gateway between the components within the cluster) [Manager]
kube-Scheduler - Looks for Pods not yet bound to a node, and assigns each Pod to a suitable node. [HR]
Kubectl - Cli tool for kubernetes. [Director]
Kubelet - Ensures that Pods are running, including their containers.(works at worker level) [Branch Manager].
Kube-proxy/Service-proxy - Maintains network rules on nodes to implement Services. (helps user to access the pods).
Container runtime - Software responsible for running containers. (Docker).
Container Network Interface (CNI Network): Provides network connectivity, ensuring that pods can communicate with each other across different nodes and that network policies are enforced. Eg- Wavenet, Calico, etc.

2. Kubernetes works in cluster

Types of Kubernetes clusters

Kubeadm - Multiple systems are connnected in a cluster by installing kubeadm on them.
Minikube (local or single ec2)
Kind cluster - kubernetes in docker
EKS/AKS/GKE

→ Kind cluster installation for linux

# For AMD64 / x86_64
[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-amd64
# For ARM64
[ $(uname -m) = aarch64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-arm64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

NOTE

Before starting to write kubernetes configs, you must know how to write yml/yaml.

3. How to Write Yet Another Markup Language(yml/yaml)

# key value
key: value
name: xyz
-----------
#list
courses:
  - list1
  - list2
  - list3
-----------
#objects
info:
  name: xyz
  age: 46
  course: CSE

Heading towards practical part…

Creating a kind cluster

We will be using yml/yaml file for creating a cluster.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  image: kindest/node:v1.31.2
- role: worker
  image: kindest/node:v1.31.2
- role: worker
  image: kindest/node:v1.31.2
- role: worker
  image: kindest/node:v1.31.2
- role: worker
  image: kindest/node:v1.31.2
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP

- Kubernetes Namespaces

|Docker container→Pod→Deployment→service |

             All this inside a namespace for isolation

A Kubernetes namespace is a mechanism for dividing and logically isolating a single physical Kubernetes cluster into multiple virtual sub-clusters.

mradul@mradul-HP-Laptop-15:~/Desktop/GIT/Fast-Translit-Deployment/Nginx$ kubectl get ns
NAME                 STATUS   AGE
default              Active   20m
kube-node-lease      Active   20m
kube-public          Active   20m
kube-system          Active   20m
local-path-storage   Active   20m
nginx                Active   4m49s
mradul@mradul-HP-Laptop-15:~/Desktop/GIT/Fast-Translit-Deployment/Nginx$

default - When no namespace is created the resources are assigned the default namespace.
kube-node-lease - Stores the information of kubernetes clusters.
kube-public - Runs the services that are publicly accessible resources.
kube-system - Runs system level services.
local-path-storage - For storage use.

- Kubernetes Pods

Pods are the smallest deployable units of computing that you can create and manage in Kubernetes. A Pod is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers.

A pod may container one or more application containers which are tightly coupled and run together.
The shared context of a Pod is a set of Linux namespaces, cgroups, and potentially other facets of isolation - the same things that isolate a container.
Pods can simply be understood by the set of containers running on same machine but still completely isolated.
Pods can even run a single container and when needed pods can also run multiple containers which are needed to run together.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

                          pods.yml

To apply the yml manifest files we use kubectl apply -f {filename} Usually we do not create pods directly using the yml file, we instead create them using the workload resources like Deployments, job, ReplicaSet, DaemonSet and StatefulSet(when we need to track the state of the pod running).
Each pod runs the single instance of the application, you must always run multiple pods for any appliation.
The replication of pods is the responsibility of the workloads and the Controllers.
Types of Multi-container pods
- Sidecar Containers: Enhance or extend the main application container, such as a separate log exporter or metrics collector.
- Init Containers: Specialized containers that execute sequentially and run to completion before any app containers start. They typically handle setup scripts, database migrations, or security credentials.
- Ephemeral Containers: Temporary troubleshooting containers that you run interactively inside an existing Pod to diagnose bugs. They cannot be added at Pod creation and lack resource guarantees.

- Kubernetes workloads

Deployment: Makes the pod scalable.

Networking for Devops

2026-09-02T05:29:25.020Z

1. How DNS actually works?

Computer and devices communicate using the IP addresses which are not easy to be remembered by a human, so they use words. Domain Name system brings these two together and gives us our destination site. Lets see how the DNS resolution takes place

We hit a request on our browser for a particular website for example xyz.com .
The browser checks its cache to find the IP address for the website xyz.com .
If the browser does not find the IP of the requested domain name, it requests the Operating System to check its cache for the IP address.
If the Operating system is also not able to find the IP address of the requested website then the, it calls the resolver.
The resolver server is usually your ISP (Internet Service Provider). All resolvers must know one thing: where to locate the root server.
The resolver checks its cache to find the address if it is already there.
If the resolver fails to find it, then the request goes to the root server.
The root server knows where to locate the .COM TLD server. TLD stands for Top-Level Domain.
The root server gives the address of the Top Level Domain server for the requested website.
There are 13 root name servers that exist today,root servers sit at the top of the DNS hierarchy. They are scattered around the globe and operated by 12 independent organisations. They are named [letter].root-servers.net where [letter] ranges from A to M.
This doesn’t mean that we have only 13 physical servers to support the whole internet, each organisation provides multiple physical servers distributed around the globe.
The Resolver now goes to the .com Top Level Domain server to find the address of the requested site.
The Top Level Domain server also does not know the IP address of the xyz.com but it will tell the name servers for that domain name, like ns1.xyz.com ns2.xyz.com ns3.xyz.com ns4.xyz.com
A nameserver is a specialized server in the Domain Name System (DNS) that acts as the internet’s phonebook. It is responsible for storing DNS records (such as IP addresses) for a domain name and answering requests to translate those human-readable names into machine-readable numerical IP addresses. Here a question arises that how did the TDL of .com domain pointed the resolver to the authoritative name servers and how could it make the connection? There are so many .COM domains!? → The answer to all these questions is Domain Registrar, when a domain is purchased the domain registrar reserves the name and communicates to the TDL registry the authoritative name servers.
Now the resolver reached the name server ns1.xyz.com there are more than one server running for a domain as they have to globally manage the load on the website. → If you want to know who are the authoritative name servers for your domain, run a WHOIS query. There are a few websites that provide this information.
The resolver get the IP of the requested domain and saves the IP address and gives it to OS.
The Operating System gives this address to the browser and then the user sees the website.

→ A STRANGE QUESTION ARISES How could the resolver find ‘ns1.xyz.com’ before ‘xyz.com’? Since ‘ns1.xyz.com’ is a subdomain of ‘xyz.com’, how could we resolve ‘ns1.xyz.com’ without resolving ‘xyz.com’ first? Isn’t the search going backwards? HOW CAN WE GET A SUB-DOMAIN WITHOUT GETTING A DOMAIN!?!?!?

The answer is GLUE RECORDS !
When the resolver asked the .COM TLD about xyz.com, extra information was attached to that response.
The resolver got at least one IP address for each name server, these are called glue records.
So the resolver not only got the name of the authoritative name server, it also got the IP address which breaks the circular dependency.

1.1 DNS Records

A Domain Name System (DNS) record is a set of instructions used to connect domain names with internet protocol (IP) addresses within DNS servers. DNS records exist as text-based files known as “zone files” written in DNS syntax. They serve as a record and set of commands on how to handle DNS queries. When a user searches a domain name or URL or takes action related to a domain name in a web browser, this is the beginning of a DNS query. A series of DNS servers then communicate with each other to resolve this query. DNS servers rely on DNS records to connect that user with the corresponding IP address and resolve all other issues. DNS records are stored on authoritative DNS servers also known as authoritative nameservers. They contain information on how frequently a server will refresh the DNS record, known as time-to-live (TTL).

Common types of DNS records:

A records: A records stands for Address records, they are the most commonly used DNS record. They directly connect the IPv4 addresses to the domain name.
AAAA records: They connect the IPv6 addresses to the domain name.
CNAME records: Canonical Name records, direct an alias domain to a canonical domain. This means that this type of record is used to link subdomains to domain A or AAAA records.
DNAME records: Delegation name records, or DNAME records, are used to redirect multiple subdomains with one record and point them to another domain. For example, a DNAME record linking domain.com to example.com will link product.domain.com, trial.domain.com, and blog.domain.com to example.com. These records are helpful in managing largescale domains and in managing domain name changes by ensuring subdomains are properly linked.
CAA records: Certification authority authorization records, or CAA records, allow domain owners to specify which certificate authorities (CAs) can issue certificates for their domain. A CA is an organization that validates the identity of websites and connects them to cryptographic keys by issuing digital certificates
CERT records: Certificate, or CERT records, store certificates that verify the authenticity of all parties involved. This type of record is particularly valuable when securing and encrypting sensitive information.
MX records: Mail exchange, or MX records, direct emails to your domain mail server. These records, along with an email server, allow for the creation of individual email accounts, such as user@example.com, that are linked to the domain (example.com)
NS records: Nameserver, or NS records, show which DNS server is acting as the authoritative nameserver for your domain. Authoritative nameservers contain the final information about a specific domain and its corresponding IP address. An NS record points to all of the different records your domain holds. Without NS records, users will not be able to access your website
SOA records: Start of authority, or SOA records, store important administrative information about a domain. This information can include the domain administrator’s email address, information on domain updates and when a server should refresh its information.
PTR records: Pointer records, or PTR records, work in the opposite direction of A records. They are used to connect an IP address with a domain name, instead of a domain name with an IP address. When a DNS lookup begins with an IP address, it then finds the corresponding hostname. These records are used to detect spam by checking if the IP addresses and associated email addresses are used by legitimate email servers. PTR records must be set up by the server host.
SPF records: Sender policy framework, or SPF records, are used to identify the mail servers that can send emails through your domain. This helps prevent your domain from being used by spammers or for malicious purposes by letting email receivers know that what they are receiving has been authorized.
SRV records: Service, or SRV records, identify a host and port for specific services, such as messaging, for a domain. Ports are virtual connection points that allow digital devices to separate different types of traffic.
ALIAS records: ALIAS records are used to direct your domain name to a host name and not an IP address. For instance, if your domain name is example.com, you can point it to product.differentexample.com using an ALIAS record.
NSEC records: Next secure records, or NSEC records, allow for proof of non-existence. This means that these records exist to confirm that other records do not exist. Being able to confirm the non-existance of a record saves time when searching for specific records.
URLFWD records: URL forwarding (or URL redirecting) is a technique used to make a single web page available via multiple URLs. (NS1 Connect) users can easily set up URL forwarding (HTTP redirects or masking) between zones. There are three types of URL redirects: Permanent (301), temporary (302), or masking.
TXT records: Text, or TXT records, store textual information related to domains and subdomains. Text records allow for the storage of SPF records and email verification records.

2. How HTTP works?

Nginx

2026-09-02T05:29:25.020Z

How to write Nginx config file

Nginx has one master process and several worker processes. The main purpose of the master process is to read and evaluate configuration, and maintain worker processes. Worker processes do actual processing of requests.
The number of worker processes is defined in the configuration file and may be fixed for a given configuration or automatically adjusted to the number of available CPU cores.
The nginx.conf file can be found in /etc/nginx or usr/local/nginx/conf or /usr/local/etc/nginx.

Config file structure

Nginx consists of modules which are controlled by directives specified in the configuration file.

NOTE

Modules provide the actual features and backend functionality, while directives are the configuration instructions you write in your configuration file to control how those modules behave.

Directives are divided into simple directives and block directives. A simple directive consists of the name and parameters separated by spaces and ends with a semicolon.
A block directive has the same structure as a simple directive, but instead of the semicolon it ends with a set of additional instructions surrounded by curly braces.
If a block directive can have other directives inside braces, it is called a context.
Directives placed in the configuration file outside of any contexts are considered to be in the main context.

Serving static content

An important web server task is serving out files.
Depending on the request, files will be served from different local directories.

http {
    server {
    }
}

                      Default config

location /www/ {
  root /data
}

This location block specifies the “/www” prefix compared with the URI from the request. For matching requests, the URI will be added to the path specified in the root directive, that is, to /data, to form the path to the requested file on the local file system. If there are several matching location blocks nginx selects the one with the longest prefix. The location block above provides the shortest prefix, of length one, and so only if all other location blocks fail to provide a match, this block will be used.

http{
    server{
        location /html/ { ;
        root /usr/share/nginx/;
        index index.html;
    }
}

This is already a working configuration of a server that listens on the standard port 80 and is accessible on the local machine at http://localhost/
Do not forget to do sudo systemctl daemon-reload after changing the config.

Nginx Reverse Proxy for an Application Running on localhost:8000

NOTE

→ Proxy - Machine or a set of machines which sit between two systems(user and a backend system or two backend systems, anything). Proxies are used to abstract the complexities or un-trusted environment. → Forward Proxy - Abstracts out the client by acting as a middleware, protects the identity of the client. Whenever the client makes a call the call goes via this proxy to the internet or to some service or machine. The service will only know the IP address of the proxy not of the client. It can be used to restrict the access of websites, at forward proxy caching can be performed as everything is going through it. → Reverse Proxy - The reverse proxy abstracts outs the complexities of the down stream systems. Example “load balancer”, Router (an api gaateway), caching, Abstraction of the infra → Firewall - A firewall is a network security device that separates a trusted internal network from an external network deemed untrustworthy, such as the internet. It regulates incoming and outgoing network traffic based on preset security rules. Firewalls are paramount in shielding networks from unauthorized access, harmful activities, and potential threats, and can exist as hardware, software, software-as-a-service (SaaS), or public or private (virtual) cloud.

Objective

Expose an application running on localhost:8000 through Nginx, allowing clients to access it via port 80.

Architecture

Client
   │
   ▼
Nginx (Port 80)
   │
   ▼
Application (localhost:8000)

Nginx acts as a reverse proxy:

Receives requests from clients.
Forwards requests to the backend application.
Returns the backend’s response to the client.

Nginx Configuration

events {}
 
http {
    server {
        listen 80;
        server_name _;
 
        location / {
            proxy_pass http://127.0.0.1:8000;
 
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

Explanation

1. Listen on Port 80

listen 80;

Tells Nginx to accept incoming HTTP requests on port 80.

2. Forward Requests to Backend

proxy_pass http://127.0.0.1:8000;

Forwards all requests received by Nginx to the application running on port 8000.

Example:

http://server-ip/
        │
        ▼
http://127.0.0.1:8000/

3. Preserve Original Request Information

Host Header

proxy_set_header Host $host;

Passes the original hostname requested by the client.

Example:

Host: example.com

Real Client IP

proxy_set_header X-Real-IP $remote_addr;

Provides the backend with the actual client IP address.

Forwarded Client Chain

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Maintains a list of proxy hops for logging and auditing.

Original Protocol

proxy_set_header X-Forwarded-Proto $scheme;

Indicates whether the client used:

http

https

Validate Configuration

Before reloading Nginx, verify the configuration syntax:

sudo nginx -t

Expected output:

nginx: configuration file syntax is ok
nginx: configuration file test is successful

Reload Nginx

Apply configuration changes:

sudo systemctl restart nginx

sudo nginx -s reload

Verify Backend Service

Check whether the application is listening on port 8000:

ss -tulnp | grep 8000

Example output:

LISTEN 0 128 127.0.0.1:8000

Test the Reverse Proxy

Open:

http://localhost

http://<server-ip>

If configured correctly:

Client → Nginx → Application → Nginx → Client

Troubleshooting

Check Nginx Error Logs

sudo tail -f /var/log/nginx/error.log

Useful for:

Backend connection failures
Invalid configurations
Permission issues

Check Access Logs

sudo tail -f /var/log/nginx/access.log

Useful for:

Verifying incoming requests
Monitoring traffic
Debugging routing issues

Key Concept

A reverse proxy hides the backend service from clients.

Clients interact with:

Nginx (Port 80)

while Nginx internally communicates with:

localhost:8000

This provides:

Centralized request handling
SSL/TLS termination
Load balancing capabilities
Security and access control
Logging and monitoring
Backend abstraction

The core directive that makes Nginx a reverse proxy is:

proxy_pass http://127.0.0.1:8000;

Everything else supports correct request forwarding and backend awareness.

Service Mesh

2026-09-02T05:29:25.020Z

What exactly a service mesh is?

Imagine you are an engineer at a fast-growing e-commerce company like Amazon. Your team started with a monolithic application—a single, large codebase handling user requests, inventory, and payments. As the company grew, you migrated to a microservices architecture with separate services for:

User Service (handles user accounts)
Order Service (processes orders)
Inventory Service (tracks stock)
Payment Service (handles payments)

At first, everything ran smoothly. However, as complexity increased, several problems emerged:

✅ Problem #0: Authentication & Authorization

Each microservice must handle logins & permissions separately, resulting in duplicated and inconsistent security logic.
🔴 Problem #1: Hard-to-Debug Failures

PopCustomers report orders failing randomly. Logs indicate errors in the Payment Service, but it’s unclear whether these issues are from network glitches, load spikes, or faulty service updates.
🔴 Problem #2: Security Risks

Sensitive data, like payment details, is transmitted between services unencrypted. Without proper encryption, interception becomes a major risk.
🔴 Problem #3: Load Balancing Issues

The Order Service is overwhelmed while the Inventory Service remains underutilized. Efficient traffic distribution is missing.
🔴 Problem #4: Slow Rollouts & Deployments

Introducing a new version of the Payment Service poses a risk. Testing it on only 10% of users is ideal, but gradual rollouts without downtime are challenging.

This growing complexity makes managing your microservices painful. This is where a Service Mesh comes in! 🚀

1. Why Do You Need a Service Mesh?

Before service meshes, organizations manually managed service-to-service communication using:

Custom code in each microservice
Reverse proxies (e.g., NGINX, HAProxy)
Load balancers
API gateways

While this approach worked, it had serious limitations:

No standardized way to handle retries, timeouts, and failures.
Security risks due to unencrypted communication.
Limited observability making it hard to trace requests across multiple services.

Real-World Example: Netflix

Netflix’s microservices faced issues like:

Slow response times because of ineffective traffic control.
DDoS threats due to the absence of a central security layer.
Authentication issues from each microservice handling its own login logic.

image.png

2. How Service Mesh Helps

image.png|102

A Service Mesh is an infrastructure layer that manages service-to-service communication within a distributed microservices architecture. It abstracts away networking concerns, thereby enhancing:

Security: Through secure, encrypted communication (mTLS).
Traffic Management: Via intelligent routing, load balancing, and retries.
Observability: With integrated logging, tracing, and monitoring.
Resilience: By using circuit breakers and fault injection.

Key benefits include decoupling networking logic from application code and centralizing security policies.

3. Service Mesh Architecture

image.png

A service mesh consists of two primary components:

3.1 Data Plane (Sidecar Proxies)

Each microservice runs alongside a sidecar proxy that intercepts all incoming and outgoing traffic.

Example: Envoy Proxy

Responsibilities:
- Service Discovery & Load Balancing: Efficiently routes traffic among service instances.
- Retries & Circuit Breaking: Enhances resilience by managing failures.
- Security Enforcement: Uses mTLS for zero-trust security.
- Telemetry Collection: Gathers metrics, logs, and traces.

3.2 Control Plane

Responsibilities:
- Manages and configures the sidecar proxies.
- Applies traffic rules, security policies, and observability settings.

These components work together to enforce consistent communication policies across your microservices.

How It Works: Without vs. With Service Mesh

Without Service Mesh:

Service A → Service B(Custom retry logic, security, and logging embedded in application code)
Service B → Database(Direct connection without security enforcement)

With Service Mesh:

Service A → Envoy Proxy (handles retries, security, monitoring) → Service B
Service B → Envoy Proxy → Database(All communications are secured and monitored)

4. Popular Service Mesh Implementations

image.png

Service Mesh	Proxy Used	Best For
Istio	Envoy	Kubernetes-native applications
Linkerd	Linkerd-proxy	Simplicity and lightweight setups
Consul	Envoy	Multi-cloud and hybrid environments
AWS App Mesh	Envoy	AWS-native applications

For more details, see the Comparison of Service Meshes.

5. Benefits of Service Mesh

5.1 Security: Encrypted & Secure Communication (mTLS)

Problem: Unencrypted traffic is vulnerable to interception.
Example: A banking app transmitting user passwords in plain text.
Solution: Enforce mTLS to ensure all communication is secure and authenticated.

kubectl apply -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: default
spec:
  mtls:
    mode: STRICT
EOF

5.2 Traffic Control

💡 Benefit: Manage how traffic flows between services, allowing canary deployments, fault injection, and load balancing.

🔀 A/B Testing with Traffic Splitting

📌 Route 80% of traffic to v1 and 20% to v2 of the reviews service

 
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
  namespace: default
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 80
    - destination:
        host: reviews
        subset: v2
      weight: 20
EOF

🔹 Now, when you access productpage, most users will see v1, while 20% get v2.

Example: Netflix gradually releases a new video recommendation algorithm.\

5.3 Observability: Debugging Made Easy

Problem: Locating the root cause of failures across multiple services is challenging.
Example: Uber’s thousands of microservices made debugging nearly impossible.
Solution: Integrate with Jaeger and Kiali for distributed tracing and visualization.

 
istioctl dashboard kiali  # Open the Kiali dashboard for service mesh visualization

5.4 Load Balancing: Efficient Traffic Distribution

Problem: Imbalanced traffic can overwhelm some services while underutilizing others.
Example: Amazon’s checkout service receives millions of requests per minute.
Solution: Dynamically distribute requests using advanced load balancing algorithms.

 
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: checkout-service
spec:
  host: checkout-service
  trafficPolicy:
    loadBalancer:
      simple: LEAST_CONN  # Directs traffic to the least busy instance

Additional Capabilities

Traffic Management: Intelligent routing, load balancing, retries, and failover.
Security Enhancements: mTLS, role-based access control (RBAC), and centralized authentication/authorization.
Observability & Monitoring: Distributed tracing (Jaeger, Zipkin), logging/metrics collection (Prometheus, Grafana, Kiali), and dynamic service discovery.
Resilience & Fault Tolerance: Circuit breaking, rate limiting, and fault injection for chaos engineering.

7. When to Use a Service Mesh?

Scenario	Use Service Mesh?
Small app with fewer than 5 services	❌ No (Overhead is too high)
10+ microservices	✅ Yes (Provides standardized communication)
Multi-cloud or hybrid deployments	✅ Yes (Simplifies cross-cloud networking)
High-security environments (banking, healthcare)	✅ Yes (Enforces mTLS and centralized authentication)

8. Real-World Benefits & Use Cases

✅ Case Study #1: Airbnb – Scaling Microservices

Problem:

Airbnb had thousands of microservices, making debugging nearly impossible due to scattered logs.
Solution:

They implemented Istio (Service Mesh) along with Jaeger for distributed tracing.
Outcome:

Engineers could trace failures in seconds, reducing incident response time by 70%.

✅ Case Study #2: Stripe – Secure Payment Transactions

Problem:

Stripe processes millions of financial transactions and required end-to-end encryption between microservices.
Solution:

They deployed a Service Mesh with mTLS to encrypt all service-to-service communication automatically.
Outcome:
- Zero unencrypted transactions
- Full compliance with banking security standards

✅ Case Study #3: Netflix – Canary Deployments

Problem:

Netflix needed to test new versions of its recommendation engine on a small subset of users to avoid downtime.
Solution:

They used Istio’s traffic splitting feature to direct 10% of traffic to the new version while maintaining 90% on the stable version.
Outcome:
- Safer rollouts
- No downtime during updates

Conclusion

A Service Mesh—using tools like Istio, Linkerd, or Consul—streamlines inter-service communication in a microservices architecture by abstracting networking, security, and observability away from application code. This approach lets developers focus on core business logic while ensuring robust, secure, and resilient interactions between services.

Feel free to reach out if you need further clarification or want to explore a live demo of these concepts in your environment!

Terraform

2026-09-02T05:29:25.020Z

prerequisite - AWS

Devops MOC

2026-09-02T05:29:25.020Z

🗺️ Devops Map of Content

Welcome to the syllabus and index for Devops.

📄 Notes

Index

2026-09-02T05:29:25.020Z

Make a note of something, Docker, System Design, Kubernetes Terraform,AWS, DBMS, Service Mesh, RabbitMQ, Testing, git-cheat-sheet.pdf, Search Engines, Postgres, GitOps, Consul, Networking for Devops, Building for Scale, OWASP, Ansible, Auth, APIs and its types, Web Security, CI CD, Advance DB concepts, FastAPI, Redis, Nginx, Infrastructure Monitoring, Python, Django, README, key.base , DSA,System Design or try the Importer!

README

2026-09-02T05:29:25.020Z

🎯 Purpose

This repository serves as:

Interview Preparation: Curated notes and explanations for common interview topics
Knowledge Base: Quick reference materials for various technologies
Learning Journal: Organized study materials as you prepare for technical interviews

📚 Directories

DevOps/

Notes on Backend & DevOps tools, methodologies, and practices including:

CI/CD pipelines
Container orchestration
Infrastructure as Code
Monitoring and logging
Cloud platforms and tools

🚀 Getting Started

Browse through the directories to find relevant topics
Each directory contains markdown files with detailed explanations
Use internal links (like [[topic]]) to navigate between related concepts
Add more directories as you expand your preparation materials

📝 Adding Content

As you add more directories and content:

Keep file names descriptive and lowercase with hyphens (e.g., docker-basics.md)
Use clear markdown formatting with headers and code blocks
Link related concepts within the vault using [[link]] syntax
Update this README to reflect new sections

🔧 Planned Additions

💡 Tips for Effective Learning

Review notes regularly before interviews
Practice explaining concepts out loud
Create examples and diagrams for complex topics
Link related concepts together for better understanding
Update notes as you learn new information

📖 How to Use

For Quick Reference: Search within markdown files for specific topics
For Deep Learning: Follow the linked concepts through the vault
For Interview Prep: Review notes in each directory before interviews

Good luck with your interview preparation! 🎓

Python

2026-09-02T05:29:25.019Z

Introduction

print("Hello world")

print(2 + 3)   # addition(+)
print(3 - 1)   # subtraction(-)
print(2 * 3)   # multiplication(*)
print(3 + 2)   # addition(+)
print(3 - 2)   # subtraction(-)
print(3 * 2)   # multiplication(*)
print(3 / 2)   # division(/)
print(3 ** 2)  # exponential(**)
print(3 % 2)   # modulus(%)
print(3 // 2)  # Floor division operator(//)

print(type(10))                  # Int
print(type(3.14))                # Float
print(type(1 + 3j))              # Complex
print(type('Asabeneh'))          # String
print(type([1, 2, 3]))           # List
print(type({'name': 'Asabeneh'}))  # Dictionary
print(type({9.8, 3.14, 2.7}))    # Set
print(type((1,2,3)))             #tuple

NOTE

Python build-in functions will be done after the functions.

Variables

Variables store data in a computer memory. Mnemonic variables are recommended to use in many programming languages. A mnemonic variable is a variable name that can be easily remembered and associated. A variable refers to a memory address in which data is stored. Number at the beginning, special character, hyphen are not allowed when naming a variable. A variable can have a short name (like x, y, z), but a more descriptive name (firstname, lastname, age, country) is highly recommended.

Python Variable Name Rules

A variable name must start with a letter or the underscore character
A variable name cannot start with a number
A variable name can only contain alpha-numeric characters and underscores (A-z, 0-9, and _ )
Variable names are case-sensitive (firstname, Firstname, FirstName and FIRSTNAME) are different variables) → If you want to use reserved keywords as the variable then use a underscore before them. Eg- “_if”

Example List of Valid variables

firstname
lastname
age
country
city
first_name
last_name
capital_city
_if # if we want to use reserved word as a variable
year_2021
year2021
current_year_2021
birth_year
num1
num2

Example list of invalid variables

first-name
first@name
first$name
num-1
1num

Declaring Multiple Variable in a Line

Multiple variables can also be declared in one line:

first_name, last_name, country, age, is_married = 'Asabeneh', 'Yetayeh', 'Helsink', 250, True

print(first_name, last_name, country, age, is_married)
print('First name:', first_name)
print('Last name: ', last_name)
print('Country: ', country)
print('Age: ', age)
print('Married: ', is_married)

Data Types

There are several data types in Python. To identify the data type we use the type built-in function.

# Different python data types
# Let's declare variables with various data types

first_name = 'Asabeneh'     # str
last_name = 'Yetayeh'       # str
country = 'Finland'         # str
city= 'Helsinki'            # str
age = 250                   # int, it is not my real age, don't worry about it

# Printing out types
print(type('Asabeneh'))          # str
print(type(first_name))          # str
print(type(10))                  # int
print(type(3.14))                # float
print(type(1 + 1j))              # complex
print(type(True))                # bool
print(type([1, 2, 3, 4]))        # list
print(type({'name':'Asabeneh'})) # dict
print(type((1,2)))               # tuple
print(type(zip([1,2],[3,4])))    # zip
print(type({1,2,3}))             # set

zip - In Python, there is no native “zip data type.” Instead, zip is a built-in constructor function that returns a specialized iterator called a zip object. This zip object maps corresponding elements from multiple iterables (like lists, tuples, or strings) into an iterator of grouped tuples.

>>> names = ["sid","panth","ani"]
>>> age = ["22","17","21"]
>>> zipped_data = zip(names,age)
>>> print(zipped_data)
<zip object at 0x7f730c059cc0>
>>> print(list(zipped_data))
[('sid', '22'), ('panth', '17'), ('ani', '21')]
>>>

Numbers in python

Number data types in Python:

Integers: Integer(negative, zero and positive) numbers Example: … -3, -2, -1, 0, 1, 2, 3 …
Floating Point Numbers(Decimal numbers) Example: … -3.5, -2.25, -1.0, 0.0, 1.1, 2.2, 3.5 …
Complex Numbers Example: 1 + j, 2 + 4j, 1 - 1j

Operators

A boolean data type represents one of the two values: True or False.

>>> if(2==1):
...   print(True)
... else:
...   print(False)
... 
False
>>>

Assignment Operators
Arithmetic Operators

Addition(+): a + b
Subtraction(-): a - b
Multiplication(*): a * b
Division(/): a / b
Modulus(%): a % b
Floor division(//): a // b
Exponentiation(**): a ** b

Comparison Operators
Logical Operators

Strings

Text is a string data type. Any data type written as text is a string. Any data under single, double or triple quote are strings. There are different string methods and built-in functions to deal with string data types. To check the length of a string use the len() method.

Creating a string

letter = 'P'                # A string could be a single character or a bunch of texts
print(letter)               # P
print(len(letter))          # 1
greeting = 'Hello, World!'  # String could be made using a single or double quote,"Hello, World!"
print(greeting)             # Hello, World!
print(len(greeting))        # 13
sentence = "I hope you are enjoying 30 days of Python Challenge"
print(sentence)

multiline_string = '''I am a teacher and enjoy teaching.
I didn't find anything as rewarding as empowering people.
That is why I created 30 days of python.'''
print(multiline_string)

# Another way of doing the same thing
multiline_string = """I am a teacher and enjoy teaching.
I didn't find anything as rewarding as empowering people.
That is why I created 30 days of python."""
print(multiline_string)

String Concatenation

first_name = 'Amitabh'
last_name = 'Bacchan'
space = ' '
full_name = first_name  +  space + last_name
print(full_name) # Amitabh Bacchan
# Checking the length of a string using len() built-in function
print(len(first_name))  # 8
print(len(last_name))   # 7
print(len(first_name) > len(last_name)) # True
print(len(full_name)) # 16

Escape Sequences in Strings In Python and other programming languages \ followed by a character is an escape sequence. Let us see the most common escape characters:
\n: new line
\t: Tab means(8 spaces)
\: Back slash
\’: Single quote (’)
\”: Double quote (”)
String Formatting

Old Style String Formatting (% Operator) %s - String (or any object with a string representation, like numbers) %d - Integers %f - Floating point numbers “%.number of digitsf” - Floating point numbers with fixed precision.

# Strings only
first_name = 'Asabeneh'
last_name = 'Yetayeh'
language = 'Python'
formated_string = 'I am %s %s. I teach %s' %(first_name, last_name, language)
print(formated_string)

# Strings  and numbers
radius = 10
pi = 3.14
area = pi * radius ** 2
formated_string = 'The area of circle with a radius %d is %.2f.' %(radius, area) # 2 refers the 2 significant digits after the point

python_libraries = ['Django', 'Flask', 'NumPy', 'Matplotlib','Pandas']
formated_string = 'The following are python libraries:%s' % (python_libraries)
print(formated_string) # "The following are python libraries:['Django', 'Flask', 'NumPy', 'Matplotlib','Pandas']"

New Style String Formatting (str.format) This format was introduced in Python version 3.

first_name = 'Asabeneh'
last_name = 'Yetayeh'
language = 'Python'
formated_string = 'I am {} {}. I teach {}'.format(first_name, last_name, language)
print(formated_string)
a = 4
b = 3

print('{} + {} = {}'.format(a, b, a + b))
print('{} - {} = {}'.format(a, b, a - b))
print('{} * {} = {}'.format(a, b, a * b))
print('{} / {} = {:.2f}'.format(a, b, a / b)) # limits it to two digits after decimal
print('{} % {} = {}'.format(a, b, a % b))
print('{} // {} = {}'.format(a, b, a // b))
print('{} ** {} = {}'.format(a, b, a ** b))

# output
4 + 3 = 7
4 - 3 = 1
4 * 3 = 12
4 / 3 = 1.33
4 % 3 = 1
4 // 3 = 1
4 ** 3 = 64

# Strings  and numbers
radius = 10
pi = 3.14
area = pi * radius ** 2
formated_string = 'The area of a circle with a radius {} is {:.2f}.'.format(radius, area) # 2 digits after decimal
print(formated_string)

String Interpolation / f-Strings (Python 3.6+) Another new string formatting is string interpolation, f-strings. Strings start with f and we can inject the data in their corresponding positions.

a = 4
b = 3
print(f'{a} + {b} = {a +b}')
print(f'{a} - {b} = {a - b}')
print(f'{a} * {b} = {a * b}')
print(f'{a} / {b} = {a / b:.2f}')
print(f'{a} % {b} = {a % b}')
print(f'{a} // {b} = {a // b}')
print(f'{a} ** {b} = {a ** b}')

Python Strings as Sequences of Characters

Python strings are sequences of characters, and share their basic methods of access with other Python ordered sequences of objects – lists and tuples. The simplest way of extracting single characters from strings (and individual members from any sequence) is to unpack them into corresponding variables.

language = 'Python'
a,b,c,d,e,f = language # unpacking sequence characters into variables
print(a) # P
print(b) # y
print(c) # t
print(d) # h
print(e) # o
print(f) # n

``` language = 'Python' first_letter = language[0] print(first_letter) # P second_letter = language[1] print(second_letter) # y last_index = len(language) - 1 last_letter = language[last_index] print(last_letter) # n language = 'Python' last_letter = language[-1] print(last_letter) # n second_last = language[-2] print(second_last) # o ```

Slicing Python Strings

language = 'Python'
first_three = language[0:3] # starts at zero index and up to 3 but not include 3
print(first_three) #Pyt
last_three = language[3:6]
print(last_three) # hon
# Another way
last_three = language[-3:]
print(last_three)   # hon
last_three = language[3:]
print(last_three)   # hon

Reversing a string

greeting = 'Hello, World!'
print(greeting[::-1]) # !dlroW ,olleH

Skipping Characters While Slicing

language = 'Python'
pto = language[0:6:2] #[starting index,ending index,steps]
print(pto) # Pto

String Methods

capitalize(): Converts the first character of the string to capital letter.

challenge = 'python is a mastikhor language'
print(challenge.capitalize()) # 'Python is a mastikhor language'

count(): returns occurrences of substring in string, count(substring, start=.., end=..). The start is a starting indexing for counting and end is the last index to count.

challenge = 'thirty days of python'
print(challenge.count('y')) # 3
print(challenge.count('y', 7, 14)) # 1, 
print(challenge.count('th')) # 2

endswith(): Checks if a string ends with a specified ending.

challenge = 'thirty days of python'
print(challenge.endswith('on'))   # True
print(challenge.endswith('tion')) # False

expandtabs(): Replaces tab character with spaces, default tab size is 8. It takes tab size argument.

challenge = 'thirty\tdays\tof\tpython'
print(challenge.expandtabs())   # 'thirty  days    of      python'
print(challenge.expandtabs(10)) # 'thirty    days      of        python'

find(): Returns the index of the first occurrence of a substring, if not found returns -1.

challenge = 'thirty days of python'
print(challenge.find('y'))  # 5
print(challenge.find('th')) # 0

rfind(): Returns the index of the last occurrence of a substring, if not found returns -1.

challenge = 'thirty days of python'
print(challenge.rfind('y'))  # 16
print(challenge.rfind('th')) # 17

index(): Returns the lowest index of a substring, additional arguments indicate starting and ending index (default 0 and string length - 1). If the substring is not found it raises a valueError.

challenge = 'thirty days of python'
sub_string = 'da'
print(challenge.index(sub_string))  # 7
print(challenge.index(sub_string, 9)) # error

rindex(): Returns the highest index of a substring, additional arguments indicate starting and ending index (default 0 and string length - 1)

challenge = 'thirty days of python'
sub_string = 'da'
print(challenge.rindex(sub_string))  # 7
print(challenge.rindex(sub_string, 9)) # error
print(challenge.rindex('on', 8)) # 19

isalnum(): Checks alphanumeric character

challenge = 'ThirtyDaysPython'
print(challenge.isalnum()) # True
 
challenge = '30DaysPython'
print(challenge.isalnum()) # True
 
challenge = 'thirty days of python'
print(challenge.isalnum()) # False, space is not an alphanumeric character
 
challenge = 'thirty days of python 2019'
print(challenge.isalnum()) # False

isalpha(): Checks if all string elements are alphabet characters (a-z and A-Z)

challenge = 'thirty days of python'
print(challenge.isalpha()) # False, space is once again excluded
challenge = 'ThirtyDaysPython'
print(challenge.isalpha()) # True
num = '123'
print(num.isalpha())      # False

isdecimal(): Checks if all characters in a string are decimal (0-9)

challenge = 'thirty days of python'
print(challenge.isdecimal())  # False
challenge = '123'
print(challenge.isdecimal())  # True
challenge = '\u00B2'
print(challenge.isdigit())   # True 
challenge = '12 3'
print(challenge.isdecimal())  # False, space not allowed

isdigit(): Checks if all characters in a string are numbers (0-9 and some other unicode characters for numbers)

challenge = 'Thirty'
print(challenge.isdigit()) # False
challenge = '30'
print(challenge.isdigit())   # True
challenge = '\u00B2'
print(challenge.isdigit())   # True

isnumeric(): Checks if all characters in a string are numbers or number related (just like isdigit(), just accepts more symbols, like ½)

num = '10'
print(num.isnumeric()) # True
num = '\u00BD' # ½
print(num.isnumeric()) # True
num = '10.5'
print(num.isnumeric()) # False

isidentifier(): Checks for a valid identifier - it checks if a string is a valid variable name

challenge = '30DaysOfPython'
print(challenge.isidentifier()) # False, because it starts with a number
challenge = 'thirty_days_of_python'
print(challenge.isidentifier()) # True

islower(): Checks if all alphabet characters in the string are lowercase

challenge = 'thirty days of python'
print(challenge.islower()) # True
challenge = 'Thirty days of python'
print(challenge.islower()) # False

isupper(): Checks if all alphabet characters in the string are uppercase

challenge = 'thirty days of python'
print(challenge.isupper()) #  False
challenge = 'THIRTY DAYS OF PYTHON'
print(challenge.isupper()) # True

join(): Returns a concatenated string

web_tech = ['HTML', 'CSS', 'JavaScript', 'React']
result = ' '.join(web_tech)
print(result) # 'HTML CSS JavaScript React'

web_tech = ['HTML', 'CSS', 'JavaScript', 'React']
result = '# '.join(web_tech)
print(result) # 'HTML# CSS# JavaScript# React'

strip(): Removes all given characters starting from the beginning and end of the string

challenge = 'thirty days of pythoonnn'
print(challenge.strip('noth')) # 'irty days of py'

replace(): Replaces substring with a given string

challenge = 'thirty days of python'
print(challenge.replace('python', 'coding')) # 'thirty days of coding'

split(): Splits the string, using given string or space as a separator

challenge = 'thirty days of python'
print(challenge.split()) # ['thirty', 'days', 'of', 'python']
challenge = 'thirty, days, of, python'
print(challenge.split(', ')) # ['thirty', 'days', 'of', 'python']

title(): Returns a title cased string

challenge = 'thirty days of python'
print(challenge.title()) # Thirty Days Of Python

swapcase(): Converts all uppercase characters to lowercase and all lowercase characters to uppercase characters

challenge = 'thirty days of python'
print(challenge.swapcase())   # THIRTY DAYS OF PYTHON
challenge = 'Thirty Days Of Python'
print(challenge.swapcase())  # tHIRTY dAYS oF pYTHON

startswith(): Checks if String Starts with the Specified String

challenge = 'thirty days of python'
print(challenge.startswith('thirty')) # True
 
challenge = '30 days of python'
print(challenge.startswith('thirty')) # False

Lists

List: is a collection which is ordered and changeable(modifiable). Allows duplicate members. A list is collection of different data types which is ordered and modifiable(mutable). A list can be empty or it may have different data type items.

Creating a list
- A list can be created in two manners → Using the build-in function list() → Using the [] square brackets

# syntax
lst = list()

empty_list = list() # this is an empty list, no item in the list
print(len(empty_list)) # 0

# syntax
lst = []

empty_list = [] # this is an empty list, no item in the list
print(len(empty_list)) # 0

fruits = ['banana', 'orange', 'mango', 'lemon']                     # list of fruits
vegetables = ['Tomato', 'Potato', 'Cabbage','Onion', 'Carrot']      # list of vegetables
animal_products = ['milk', 'meat', 'butter', 'yoghurt']             # list of animal products
web_techs = ['HTML', 'CSS', 'JS', 'React','Redux', 'Node', 'MongDB'] # list of web technologies
countries = ['Finland', 'Estonia', 'Denmark', 'Sweden', 'Norway'] 

# Print the lists and its length
print('Fruits:', fruits)
print('Number of fruits:', len(fruits))
print('Vegetables:', vegetables)
print('Number of vegetables:', len(vegetables))
print('Animal products:',animal_products)
print('Number of animal products:', len(animal_products))
print('Web technologies:', web_techs)
print('Number of web technologies:', len(web_techs))
print('Countries:', countries)
print('Number of countries:', len(countries))

Outputs:

output
Fruits: ['banana', 'orange', 'mango', 'lemon']
Number of fruits: 4
Vegetables: ['Tomato', 'Potato', 'Cabbage', 'Onion', 'Carrot']
Number of vegetables: 5
Animal products: ['milk', 'meat', 'butter', 'yoghurt']
Number of animal products: 4
Web technologies: ['HTML', 'CSS', 'JS', 'React', 'Redux', 'Node', 'MongDB']
Number of web technologies: 7
Countries: ['Finland', 'Estonia', 'Denmark', 'Sweden', 'Norway']
Number of countries: 5

A list can have more than one datatype

 lst = ['Asabeneh', 250, True, {'country':'Finland', 'city':'Helsinki'}] # list containing different data types

Indexing in list

Unpacking list items

lst = ['item1','item2','item3', 'item4', 'item5']
first_item, second_item, third_item, *rest = lst
print(first_item)     # item1
print(second_item)    # item2
print(third_item)     # item3
print(rest)           # ['item4', 'item5']

# First Example
fruits = ['banana', 'orange', 'mango', 'lemon','lime','apple']
first_fruit, second_fruit, third_fruit, *rest = fruits 
print(first_fruit)     # banana
print(second_fruit)    # orange
print(third_fruit)     # mango
print(rest)           # ['lemon','lime','apple']
# Second Example about unpacking list
first, second, third,*rest, tenth = [1,2,3,4,5,6,7,8,9,10]
print(first)          # 1
print(second)         # 2
print(third)          # 3
print(rest)           # [4,5,6,7,8,9]
print(tenth)          # 10
# Third Example about unpacking list
countries = ['Germany', 'France','Belgium','Sweden','Denmark','Finland','Norway','Iceland','Estonia']
gr, fr, bg, sw, *scandic, es = countries
print(gr) 
print(fr)
print(bg)
print(sw)
print(scandic)
print(es)

Slicing Items from a List Positive Indexing: We can specify a range of positive indexes by specifying the start, end and step, the return value will be a new list. (default values for start = 0, end = len(lst) - 1 (last item), step = 1)

fruits = ['banana', 'orange', 'mango', 'lemon']
all_fruits = fruits[0:4] # it returns all the fruits
# this will also give the same result as the one above
all_fruits = fruits[0:] # if we don't set where to stop it takes all the rest
orange_and_mango = fruits[1:3] # it does not include the first index
orange_mango_lemon = fruits[1:]
orange_and_lemon = fruits[::2] # here we used a 3rd argument, step. It will take every 2cnd item - ['banana', 'mango']

Negative Indexing: We can specify a range of negative indexes by specifying the start, end and step, the return value will be a new list.

fruits = ['banana', 'orange', 'mango', 'lemon']
all_fruits = fruits[-4:] # it returns all the fruits
orange_and_mango = fruits[-3:-1] # it does not include the last index,['orange', 'mango']
orange_mango_lemon = fruits[-3:] # this will give starting from -3 to the end,['orange', 'mango', 'lemon']
reverse_fruits = fruits[::-1] # a negative step will take the list in reverse order,['lemon', 'mango', 'orange', 'banana']

Modifying Lists List is a mutable or modifiable ordered collection of items. Lets modify the fruit list.

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits[0] = 'avocado'
print(fruits)       #  ['avocado', 'orange', 'mango', 'lemon']
fruits[1] = 'apple'
print(fruits)       #  ['avocado', 'apple', 'mango', 'lemon']
last_index = len(fruits) - 1
fruits[last_index] = 'lime'
print(fruits)        #  ['avocado', 'apple', 'mango', 'lime']

Checking Items in a List Checking an item if it is a member of a list using in operator. See the example below.

fruits = ['banana', 'orange', 'mango', 'lemon']
does_exist = 'banana' in fruits
print(does_exist)  # True
does_exist = 'lime' in fruits
print(does_exist)  # False

Adding Items to a List To add item to the end of an existing list we use the method append().

# syntax
lst = list()
lst.append(item)

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.append('apple')
print(fruits)           # ['banana', 'orange', 'mango', 'lemon', 'apple']
fruits.append('lime')   # ['banana', 'orange', 'mango', 'lemon', 'apple', 'lime']
print(fruits)

Inserting Items into a List We can use insert() method to insert a single item at a specified index in a list. Note that other items are shifted to the right. The insert() methods takes two arguments:index and an item to insert.

# syntax
lst = ['item1', 'item2']
lst.insert(index, item)

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.insert(2, 'apple') # insert apple between orange and mango
print(fruits)           # ['banana', 'orange', 'apple', 'mango', 'lemon']
fruits.insert(3, 'lime')   # ['banana', 'orange', 'apple', 'lime', 'mango', 'lemon']
print(fruits)

Removing Items from a List The remove method removes a specified item from a list

# syntax
lst = ['item1', 'item2']
lst.remove(item)

fruits = ['banana', 'orange', 'mango', 'lemon', 'banana']
fruits.remove('banana')
print(fruits)  # ['orange', 'mango', 'lemon', 'banana'] - this method removes the first occurrence of the item in the list
fruits.remove('lemon')
print(fruits)  # ['orange', 'mango', 'banana']

Removing Items Using Pop The pop() method removes the specified index, (or the last item if index is not specified):

# syntax
lst = ['item1', 'item2']
lst.pop()       # last item
lst.pop(index)

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.pop()
print(fruits)       # ['banana', 'orange', 'mango']
 
fruits.pop(0)
print(fruits)       # ['orange', 'mango']

Removing Items Using Del The del keyword removes the specified index and it can also be used to delete items within index range. It can also delete the list completely

# syntax
lst = ['item1', 'item2']
del lst[index] # only a single item
del lst        # to delete the list completely

fruits = ['banana', 'orange', 'mango', 'lemon', 'kiwi', 'lime']
del fruits[0]
print(fruits)       # ['orange', 'mango', 'lemon', 'kiwi', 'lime']
del fruits[1]
print(fruits)       # ['orange', 'lemon', 'kiwi', 'lime']
del fruits[1:3]     # this deletes items between given indexes, so it does not delete the item with index 3!
print(fruits)       # ['orange', 'lime']
del fruits
print(fruits)       # This should give: NameError: name 'fruits' is not defined

Clearing List Items The clear() method empties the list:

# syntax
lst = ['item1', 'item2']
lst.clear()

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.clear()
print(fruits)       # []

Copying a List It is possible to copy a list by reassigning it to a new variable in the following way: list2 = list1. Now, list2 is a reference of list1, any changes we make in list2 will also modify the original, list1. But there are lots of case in which we do not like to modify the original instead we like to have a different copy. One of way of avoiding the problem above is using copy().

# syntax
lst = ['item1', 'item2']
lst_copy = lst.copy()

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits_copy = fruits.copy()
print(fruits_copy)       # ['banana', 'orange', 'mango', 'lemon']

Joining Lists There are several ways to join, or concatenate, two or more lists in Python. → Plus Operator (+)

# syntax
list3 = list1 + list2

positive_numbers = [1, 2, 3, 4, 5]
zero = [0]
negative_numbers = [-5,-4,-3,-2,-1]
integers = negative_numbers + zero + positive_numbers
print(integers) # [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5]
fruits = ['banana', 'orange', 'mango', 'lemon']
vegetables = ['Tomato', 'Potato', 'Cabbage', 'Onion', 'Carrot']
fruits_and_vegetables = fruits + vegetables
print(fruits_and_vegetables ) # ['banana', 'orange', 'mango', 'lemon', 'Tomato', 'Potato', 'Cabbage', 'Onion', 'Carrot']

→ Joining using extend() method The extend() method allows to append list in a list. See the example below.

# syntax
list1 = ['item1', 'item2']
list2 = ['item3', 'item4', 'item5']
list1.extend(list2) # ['item1', 'item2', 'item3', 'item4', 'item5']

num1 = [0, 1, 2, 3]
num2= [4, 5, 6]
num1.extend(num2)
print('Numbers:', num1) # Numbers: [0, 1, 2, 3, 4, 5, 6]
negative_numbers = [-5,-4,-3,-2,-1]
positive_numbers = [1, 2, 3,4,5]
zero = [0]
negative_numbers.extend(zero)
negative_numbers.extend(positive_numbers)
print('Integers:', negative_numbers) # Integers: [-5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5]
fruits = ['banana', 'orange', 'mango', 'lemon']
vegetables = ['Tomato', 'Potato', 'Cabbage', 'Onion', 'Carrot']
fruits.extend(vegetables)
print('Fruits and vegetables:', fruits ) # Fruits and vegetables: ['banana', 'orange', 'mango', 'lemon', 'Tomato', 'Potato', 'Cabbage', 'Onion', 'Carrot']

Counting Items in a List The count() method returns the number of times an item appears in a list:

# syntax
lst = ['item1', 'item2']
lst.count(item)

fruits = ['banana', 'orange', 'mango', 'lemon']
print(fruits.count('orange'))   # 1
ages = [22, 19, 24, 25, 26, 24, 25, 24]
print(ages.count(24))           # 3

Finding Index of an Item The index() method returns the index of an item in the list:

# syntax
lst = ['item1', 'item2']
lst.index(item)

fruits = ['banana', 'orange', 'mango', 'lemon']
print(fruits.index('orange'))   # 1
ages = [22, 19, 24, 25, 26, 24, 25, 24]
print(ages.index(24))           # 2, the first occurrence

Reversing a List The reverse() method reverses the order of a list.

# syntax
lst = ['item1', 'item2']
lst.reverse()

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.reverse()
print(fruits) # ['lemon', 'mango', 'orange', 'banana']
ages = [22, 19, 24, 25, 26, 24, 25, 24]
ages.reverse()
print(ages) # [24, 25, 24, 26, 25, 24, 19, 22]

Sorting List Items To sort lists we can use sort() method or sorted() built-in functions. The sort() method reorders the list items in ascending order and modifies the original list. If an argument of sort() method reverse is equal to true, it will arrange the list in descending order. → sort(): this method modifies the original list

# syntax
lst = ['item1', 'item2']
lst.sort()                # ascending
lst.sort(reverse=True)    # descending

Example:

fruits = ['banana', 'orange', 'mango', 'lemon']
fruits.sort()
print(fruits)             # sorted in alphabetical order, ['banana', 'lemon', 'mango', 'orange']
fruits.sort(reverse=True)
print(fruits) # ['orange', 'mango', 'lemon', 'banana']
ages = [22, 19, 24, 25, 26, 24, 25, 24]
ages.sort()
print(ages) #  [19, 22, 24, 24, 24, 25, 25, 26]
 
ages.sort(reverse=True)
print(ages) #  [26, 25, 25, 24, 24, 24, 22, 19]

sorted(): returns the ordered list without modifying the original list Example:

fruits = ['banana', 'orange', 'mango', 'lemon']
print(sorted(fruits))   # ['banana', 'lemon', 'mango', 'orange']
# Reverse order
fruits = ['banana', 'orange', 'mango', 'lemon']
fruits = sorted(fruits,reverse=True)
print(fruits)     # ['orange', 'mango', 'lemon', 'banana']

Tuples

A tuple is a collection of different data types which is ordered and unchangeable (immutable). Tuples are written with round brackets, (). Once a tuple is created, we cannot change its values. We cannot use add, insert, remove methods in a tuple because it is not modifiable (mutable). Unlike list, tuple has few methods. Methods related to tuples:

tuple(): to create an empty tuple
count(): to count the number of a specified item in a tuple
index(): to find the index of a specified item in a tuple
+ operator: to join two or more tuples and to create a new tuple

Creating a Tuple

Empty tuple: Creating an empty tuple

# syntax
empty_tuple = ()
# or using the tuple constructor
empty_tuple = tuple()

Tuple with initial values

# syntax
tpl = ('item1', 'item2','item3')

fruits = ('banana', 'orange', 'mango', 'lemon')

Tuple length

We use the len() method to get the length of a tuple.

# syntax
tpl = ('item1', 'item2', 'item3')
len(tpl)

Accessing Tuple Items

Positive Indexing Similar to the list data type we use positive or negative indexing to access tuple items.

# Syntax
tpl = ('item1', 'item2', 'item3')
first_item = tpl[0]
second_item = tpl[1]

fruits = ('banana', 'orange', 'mango', 'lemon')
first_fruit = fruits[0]
second_fruit = fruits[1]
last_index =len(fruits) - 1
last_fruit = fruits[last_index]

Negative indexing Negative indexing means beginning from the end, -1 refers to the last item, -2 refers to the second last and the negative of the list/tuple length refers to the first item.

# Syntax
tpl = ('item1', 'item2', 'item3','item4')
first_item = tpl[-4]
second_item = tpl[-3]

fruits = ('banana', 'orange', 'mango', 'lemon')
first_fruit = fruits[-4]
second_fruit = fruits[-3]
last_fruit = fruits[-1]

Slicing tuples

We can slice out a sub-tuple by specifying a range of indexes where to start and where to end in the tuple, the return value will be a new tuple with the specified items.

Range of Positive Indexes

# Syntax
tpl = ('item1', 'item2', 'item3','item4')
all_items = tpl[0:4]         # all items
all_items = tpl[0:]         # all items
middle_two_items = tpl[1:3]  # does not include item at index 3

fruits = ('banana', 'orange', 'mango', 'lemon')
all_fruits = fruits[0:4]    # all items
all_fruits= fruits[0:]      # all items
orange_mango = fruits[1:3]  # doesn't include item at index 3
orange_to_the_rest = fruits[1:]

Range of Negative Indexes

# Syntax
tpl = ('item1', 'item2', 'item3','item4')
all_items = tpl[-4:]         # all items
middle_two_items = tpl[-3:-1]  # does not include item at index 3 (-1)

fruits = ('banana', 'orange', 'mango', 'lemon')
all_fruits = fruits[-4:]    # all items
orange_mango = fruits[-3:-1]  # doesn't include item at index 3
orange_to_the_rest = fruits[-3:]

Changing Tuples to Lists

We can change tuples to lists and lists to tuples. Tuple is immutable if we want to modify a tuple we should change it to a list.

# Syntax
tpl = ('item1', 'item2', 'item3','item4')
lst = list(tpl)

fruits = ('banana', 'orange', 'mango', 'lemon')
fruits = list(fruits)
fruits[0] = 'apple'
print(fruits)     # ['apple', 'orange', 'mango', 'lemon']
fruits = tuple(fruits)
print(fruits)     # ('apple', 'orange', 'mango', 'lemon')

Checking an Item in a Tuple

We can check if an item exists or not in a tuple using in, it returns a boolean.

# Syntax
tpl = ('item1', 'item2', 'item3','item4')
'item2' in tpl # True

fruits = ('banana', 'orange', 'mango', 'lemon')
print('orange' in fruits) # True
print('apple' in fruits) # False
fruits[0] = 'apple' # TypeError: 'tuple' object does not support item assignment

Joining Tuples

We can join two or more tuples using + operator

# syntax
tpl1 = ('item1', 'item2', 'item3')
tpl2 = ('item4', 'item5','item6')
tpl3 = tpl1 + tpl2

fruits = ('banana', 'orange', 'mango', 'lemon')
vegetables = ('Tomato', 'Potato', 'Cabbage','Onion', 'Carrot')
fruits_and_vegetables = fruits + vegetables

Deleting Tuples

It is not possible to remove a single item in a tuple but it is possible to delete the tuple itself using del.

# syntax
tpl1 = ('item1', 'item2', 'item3')
del tpl1

fruits = ('banana', 'orange', 'mango', 'lemon')
del fruits

Sets

Set is a collection of items. Let me take you back to your elementary or high school Mathematics lesson. The Mathematics definition of a set can be applied also in Python. Set is a collection of unordered and un-indexed distinct elements. In Python set is used to store unique items, and it is possible to find the union, intersection, difference, symmetric difference, subset, super set and disjoint set among sets.

Creating a Set

To create an empty set, we use the set() function. Empty curly brackets {} will create a dictionary.

Creating an empty set

# syntax
st = set()

Creating a set with initial items

# syntax
st = {'item1', 'item2', 'item3', 'item4'}

Example:

# syntax
fruits = {'banana', 'orange', 'mango', 'lemon'}

Getting Set’s Length

We use len() method to find the length of a set.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
len(st)

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
len(fruits)

Accessing Items in a Set

We use loops to access items. We will see this in loop section

Checking an Item

To check if an item exist in a list we use in membership operator.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
print("Does set st contain item3? ", 'item3' in st) # Does set st contain item3? True

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
print('mango' in fruits ) # True

Adding Items to a Set

Once a set is created we cannot change any items and we can also add additional items.

Add one item using add()

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
st.add('item5')

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
fruits.add('lime')

Add multiple items using update() The update() allows to add multiple items to a set. The update() takes a list argument.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
st.update(['item5','item6','item7'])

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
vegetables = ('tomato', 'potato', 'cabbage','onion', 'carrot')
fruits.update(vegetables)

Removing Items from a Set

We can remove an item from a set using remove() method. If the item is not found remove() method will raise errors, so it is good to check if the item exist in the given set. However, discard() method doesn’t raise any errors.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
st.remove('item2')

The pop() methods remove a random item from a list and it returns the removed item. Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
fruits.pop()  # removes a random item from the set

If we are interested in the removed item.

fruits = {'banana', 'orange', 'mango', 'lemon'}
removed_item = fruits.pop()

Clearing Items in a Set

If we want to clear or empty the set we use clear method.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
st.clear()

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
fruits.clear()
print(fruits) # set()

Deleting a Set

If we want to delete the set itself we use del operator.

# syntax
st = {'item1', 'item2', 'item3', 'item4'}
del st

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
del fruits

Converting List to Set

We can convert list to set and set to list. Converting list to set removes duplicates and only unique items will be reserved.

# syntax
lst = ['item1', 'item2', 'item3', 'item4', 'item1']
st = set(lst)  # {'item2', 'item4', 'item1', 'item3'} - the order is random, because sets in general are unordered

Example:

fruits = ['banana', 'orange', 'mango', 'lemon','orange', 'banana']
fruits = set(fruits) # {'mango', 'lemon', 'banana', 'orange'}

Joining Sets

We can join two sets using the union() or update() method or | symbol .

Union This method returns a new set

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item5', 'item6', 'item7', 'item8'}
st3 = st1.union(st2) #st3 = st1 | st2

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
vegetables = {'tomato', 'potato', 'cabbage','onion', 'carrot'}
print(fruits.union(vegetables)) # {'lemon', 'carrot', 'tomato', 'banana', 'mango', 'orange', 'cabbage', 'potato', 'onion'}
# or using this : print(fruits | vegetables)

Update This method inserts a set into a given set

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item5', 'item6', 'item7', 'item8'}
st1.update(st2) # st2 contents are added to st1

Example:

fruits = {'banana', 'orange', 'mango', 'lemon'}
vegetables = {'tomato', 'potato', 'cabbage','onion', 'carrot'}
fruits.update(vegetables)
print(fruits) # {'lemon', 'carrot', 'tomato', 'banana', 'mango', 'orange', 'cabbage', 'potato', 'onion'}

Finding Intersection Items

Intersection returns a set of items which are in both the sets or using & symbol. See the example

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item3', 'item2'}
st1.intersection(st2) # {'item3', 'item2'}
# or using thia : st1 & st2

Example:

whole_numbers = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
even_numbers = {0, 2, 4, 6, 8, 10}
whole_numbers.intersection(even_numbers) # {0, 2, 4, 6, 8, 10}
 
python = {'p', 'y', 't', 'h', 'o','n'}
dragon = {'d', 'r', 'a', 'g', 'o','n'}
python.intersection(dragon)     # {'o', 'n'}
# python & dragon

Checking Subset and Super Set

A set can be a subset or super set of other sets:

Subset: issubset()
Super set: issuperset

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item2', 'item3'}
st2.issubset(st1) # True
st1.issuperset(st2) # True

Example:

whole_numbers = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
even_numbers = {0, 2, 4, 6, 8, 10}
whole_numbers.issubset(even_numbers) # False, because it is a super set
whole_numbers.issuperset(even_numbers) # True
 
python = {'p', 'y', 't', 'h', 'o','n'}
dragon = {'d', 'r', 'a', 'g', 'o','n'}
python.issubset(dragon)     # False

Checking the Difference Between Two Sets

It returns the difference between two sets or using - symbol .

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item2', 'item3'}
st2.difference(st1) # set() : st2 - st1
st1.difference(st2) # {'item1', 'item4'} => st1\st2  : st2 - st1

Example:

whole_numbers = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
even_numbers = {0, 2, 4, 6, 8, 10}
whole_numbers.difference(even_numbers) # {1, 3, 5, 7, 9}
 
python = {'p', 'y', 't', 'o','n'}
dragon = {'d', 'r', 'a', 'g', 'o','n'}
python.difference(dragon)     # {'p', 'y', 't'}  - the result is unordered (characteristic of sets)
# python - dragon
dragon.difference(python)     # {'d', 'r', 'a', 'g'}
# dragon - python

Finding Symmetric Difference Between Two Sets

It returns the symmetric difference between two sets. It means that it returns a set that contains all items from both sets, except items that are present in both sets, mathematically: (A\B) ∪ (B\A)

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item2', 'item3'}
# it means (A\B)∪(B\A)
st2.symmetric_difference(st1) # {'item1', 'item4'} : st2 ^ st1

Example:

whole_numbers = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
some_numbers = {1, 2, 3, 4, 5}
whole_numbers.symmetric_difference(some_numbers) # {0, 6, 7, 8, 9, 10}
 
python = {'p', 'y', 't', 'h', 'o','n'}
dragon = {'d', 'r', 'a', 'g', 'o','n'}
python.symmetric_difference(dragon)  # {'r', 't', 'p', 'y', 'g', 'a', 'd', 'h'}
# python ^ dragon

Joining Sets

If two sets do not have a common item or items we call them disjoint sets. We can check if two sets are joint or disjoint using isdisjoint() method.

# syntax
st1 = {'item1', 'item2', 'item3', 'item4'}
st2 = {'item2', 'item3'}
st2.isdisjoint(st1) # False

Example:

even_numbers = {0, 2, 4 ,6, 8}
odd_numbers = {1, 3, 5, 7, 9}
even_numbers.isdisjoint(odd_numbers) # True, because no common item
 
python = {'p', 'y', 't', 'h', 'o','n'}
dragon = {'d', 'r', 'a', 'g', 'o','n'}
python.isdisjoint(dragon)  # False, there are common items {'o', 'n'}

Dictionaries

A dictionary is a collection of unordered, modifiable(mutable) paired (key: value) data type.

Creating a Dictionary

To create a dictionary we use curly brackets, {} or the dict() built-in function.

# syntax
empty_dict = {}
# Dictionary with data values
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }

The dictionary above shows that a value could be any data types:string, boolean, list, tuple, set or a dictionary.

Dictionary Length

It checks the number of ‘key: value’ pairs in the dictionary.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
print(len(dct)) # 4

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_married':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }
print(len(person)) # 7

Accessing Dictionary Items

We can access Dictionary items by referring to its key name.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
print(dct['key1']) # value1
print(dct['key4']) # value4

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }
print(person['first_name']) # Asabeneh
print(person['country'])    # Finland
print(person['skills'])     # ['JavaScript', 'React', 'Node', 'MongoDB', 'Python']
print(person['skills'][0])  # JavaScript
print(person['address']['street']) # Space street
print(person['city'])       # Error

Accessing an item by key name raises an error if the key does not exist. To avoid this error first we have to check if a key exist or we can use the get method. The get method returns None, which is a NoneType object data type, if the key does not exist.

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }
print(person.get('first_name')) # Asabeneh
print(person.get('country'))    # Finland
print(person.get('skills')) #['JavaScript', 'React', 'Node', 'MongoDB', 'Python']
print(person.get('city'))   # None

Adding Items to a Dictionary

We can add new key and value pairs to a dictionary

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
dct['key5'] = 'value5'

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
        }
}
person['job_title'] = 'Instructor'
person['skills'].append('HTML')
print(person)

Modifying Items in a Dictionary

We can modify items in a dictionary

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
dct['key1'] = 'value-one'

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }
person['first_name'] = 'Eyob'
person['age'] = 252

Checking Keys in a Dictionary

We use the in operator to check if a key exist in a dictionary

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
print('key2' in dct) # True
print('key5' in dct) # False

Removing Key and Value Pairs from a Dictionary

pop(key): removes the item with the specified key name:
popitem(): removes the last item
del: removes an item with specified key name

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
dct.pop('key1') # removes key1 item
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
dct.popitem() # removes the last item
del dct['key2'] # removes key2 item

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
    }
person.pop('first_name')        # Removes the firstname item
person.popitem()                # Removes the address item
del person['is_married']        # Removes the is_married item

Changing Dictionary to a List of Items

The items() method changes dictionary to a list of tuples.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
print(dct.items()) # dict_items([('key1', 'value1'), ('key2', 'value2'), ('key3', 'value3'), ('key4', 'value4')])

Clearing a Dictionary

If we don’t want the items in a dictionary we can clear them using clear() method

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
print(dct.clear()) # None

Deleting a Dictionary

If we do not use the dictionary we can delete it completely

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
del dct

Copy a Dictionary

We can copy a dictionary using a copy() method. Using copy we can avoid mutation of the original dictionary.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
dct_copy = dct.copy() # {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}

Getting Dictionary Keys as a List

The keys() method gives us all the keys of a a dictionary as a list.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
keys = dct.keys()
print(keys)     # dict_keys(['key1', 'key2', 'key3', 'key4'])

Getting Dictionary Values as a List

The values method gives us all the values of a a dictionary as a list.

# syntax
dct = {'key1':'value1', 'key2':'value2', 'key3':'value3', 'key4':'value4'}
values = dct.values()
print(values)     # dict_values(['value1', 'value2', 'value3', 'value4'])

Conditionals

By default, statements in Python script are executed sequentially from top to bottom. If the processing logic require so, the sequential flow of execution can be altered in two way:

Conditional execution: a block of one or more statements will be executed if a certain expression is true
Repetitive execution: a block of one or more statements will be repetitively executed as long as a certain expression is true. In this section, we will cover if, else, elif statements. The comparison and logical operators we learned in previous sections will be useful here.

If Condition

In python and other programming languages the key word if is used to check if a condition is true and to execute the block code. Remember the indentation after the colon.

# syntax
if condition:
    this part of code runs for truthy conditions

Example: 1

a = 3
if a > 0:
    print('A is a positive number')
# A is a positive number

As you can see in the example above, 3 is greater than 0. The condition was true and the block code was executed. However, if the condition is false, we do not see the result. In order to see the result of the falsy condition, we should have another block, which is going to be else.

If Else

If condition is true the first block will be executed, if not the else condition will run.

# syntax
if condition:
    this part of code runs for truthy conditions
else:
     this part of code runs for false conditions

Example:

a = 3
if a < 0:
    print('A is a negative number')
else:
    print('A is a positive number')

The condition above proves false, therefore the else block was executed. How about if our condition is more than two? We could use elif.

If Elif Else

In our daily life, we make decisions on daily basis. We make decisions not by checking one or two conditions but multiple conditions. As similar to life, programming is also full of conditions. We use elif when we have multiple conditions.

# syntax
if condition:
    code
elif condition:
    code
else:
    code

Example:

a = 0
if a > 0:
    print('A is a positive number')
elif a < 0:
    print('A is a negative number')
else:
    print('A is zero')

Short Hand

# syntax
code if condition else code

Example:

a = 3
print('A is positive') if a > 0 else print('A is negative') # first condition met, 'A is positive' will be printed

Nested Conditions

Conditions can be nested

# syntax
if condition:
    code
    if condition:
    code

Example:

a = 0
if a > 0:
    if a % 2 == 0:
        print('A is a positive and even integer')
    else:
        print('A is a positive number')
elif a == 0:
    print('A is zero')
else:
    print('A is a negative number')

We can avoid writing nested condition by using logical operator and.

If Condition and Logical Operators

# syntax
if condition and condition:
    code

Example:

a = 0
if a > 0 and a % 2 == 0:
        print('A is an even and positive integer')
elif a > 0 and a % 2 !=  0:
     print('A is a positive integer')
elif a == 0:
    print('A is zero')
else:
    print('A is negative')

If and Or Logical Operators

# syntax
if condition or condition:
    code

Example:

user = 'James'
access_level = 3
if user == 'admin' or access_level >= 4:
        print('Access granted!')
else:
    print('Access denied!')

Loops

Life is full of routines. In programming we also do lots of repetitive tasks. In order to handle repetitive task programming languages use loops. Python programming language also provides the following types of two loops:

while loop
for loop

While Loop

We use the reserved word while to make a while loop. It is used to execute a block of statements repeatedly until a given condition is satisfied. When the condition becomes false, the lines of code after the loop will be continued to be executed.

  # syntax
while condition:
    code goes here

Example:

count = 0
while count < 5:
    print(count)
    count = count + 1
#prints from 0 to 4

In the above while loop, the condition becomes false when count is 5. That is when the loop stops. If we are interested to run block of code once the condition is no longer true, we can use else.

  # syntax
while condition:
    code goes here
else:
    code goes here

Example:

count = 0
while count < 5:
    print(count)
    count = count + 1
else:
    print(count)

The above loop condition will be false when count is 5 and the loop stops, and execution starts the else statement. As a result 5 will be printed.

Break and Continue - Part 1

Break: We use break when we like to get out of or stop the loop.

# syntax
while condition:
    code goes here
    if another_condition:
        break

Example:

count = 0
while count < 5:
    print(count)
    count = count + 1
    if count == 3:
        break

The above while loop only prints 0, 1, 2, but when it reaches 3 it stops.

Continue: With the continue statement we can skip the current iteration, and continue with the next:

  # syntax
while condition:
    code goes here
    if another_condition:
        continue

Example:

count = 0
while count < 5:
    if count == 3:
        count += 1
        continue
    print(count)
    count = count + 1

The above while loop only prints 0, 1, 2 and 4 (skips 3).

For Loop

A for keyword is used to make a for loop, similar with other programming languages, but with some syntax differences. Loop is used for iterating over a sequence (that is either a list, a tuple, a dictionary, a set, or a string).

-Using For loop on list

# syntax
for iterator in lst:
    code goes here

Example:

numbers = [0, 1, 2, 3, 4, 5]
for number in numbers: # number is temporary name to refer to the list's items, valid only inside this loop
    print(number)       # the numbers will be printed line by line, from 0 to 5

-Using For loop on string

# syntax
for iterator in string:
    code goes here

Example:

language = 'Python'
for letter in language:
    print(letter)
 
 
for i in range(len(language)):
    print(language[i])

-Using For loop on tuple

# syntax
for iterator in tpl:
    code goes here

Example:

numbers = (0, 1, 2, 3, 4, 5)
for number in numbers:
    print(number)

For loop with dictionary Looping through a dictionary gives you the key of the dictionary.

  # syntax
for iterator in dct:
    code goes here

Example:

person = {
    'first_name':'Asabeneh',
    'last_name':'Yetayeh',
    'age':250,
    'country':'Finland',
    'is_marred':True,
    'skills':['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address':{
        'street':'Space street',
        'zipcode':'02210'
    }
}
for key in person:
    print(key)
 
for key, value in person.items():
    print(key, value) # this way we get both keys and values printed out

-Using For Loop in set

# syntax
for iterator in st:
    code goes here

Example:

it_companies = {'Facebook', 'Google', 'Microsoft', 'Apple', 'IBM', 'Oracle', 'Amazon'}
for company in it_companies:
    print(company)

Break and Continue - Part 2

Short reminder: Break: We use break when we want to stop our loop before it is completed.

# syntax
for iterator in sequence:
    code goes here
    if condition:
        break

Example:

numbers = (0,1,2,3,4,5)
for number in numbers:
    print(number)
    if number == 3:
        break

In the above example, the loop stops when it reaches 3.

Continue: We use continue when we want to skip some of the steps in the iteration of the loop.

  # syntax
for iterator in sequence:
    code goes here
    if condition:
        continue

Example:

numbers = (0,1,2,3,4,5)
for number in numbers:
    print(number)
    if number == 3:
        continue
    print('Next number should be ', number + 1) if number != 5 else print("loop's end") # for short hand conditions need both if and else statements
print('outside the loop')

In the example above, if the number equals 3, the step after the condition (but inside the loop) is skipped and the execution of the loop continues if there are any iterations left.

The Range Function

The range() function is used to return a list of numbers. The range(start, end, step) takes three parameters: starting, ending and increment. By default it starts from 0 and the increment is 1. The range sequence needs at least 1 argument (end). Creating sequences using range

lst = list(range(11))
print(lst) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
st = set(range(1, 11))    # 2 arguments indicate start and end of the sequence, step set to default 1
print(st) # {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
 
lst = list(range(0,11,2))
print(lst) # [0, 2, 4, 6, 8, 10]
st = set(range(0,11,2))
print(st) #  {0, 2, 4, 6, 8, 10}
 
# for backward from start to end 
lst = list(range(11,0,-2))
print(lst) # [11,9,7,5,3,1]

# syntax
for iterator in range(start, end, step):

Example:

for number in range(11):
    print(number)   # prints 0 to 10, not including 11

Nested For Loop

We can write loops inside a loop.

# syntax
for x in y:
    for t in x:
        print(t)

Example:

person = {
    'first_name': 'Asabeneh',
    'last_name': 'Yetayeh',
    'age': 250,
    'country': 'Finland',
    'is_marred': True,
    'skills': ['JavaScript', 'React', 'Node', 'MongoDB', 'Python'],
    'address': {
        'street': 'Space street',
        'zipcode': '02210'
    }
}
for key in person:
    if key == 'skills':
        for skill in person['skills']:
            print(skill)

For Else

If we want to execute some message when the loop ends, we use else.

# syntax
for iterator in range(start, end, step):
    do something
else:
    print('The loop ended')

Example:

for number in range(11):
    print(number)   # prints 0 to 10, not including 11
else:
    print('The loop stops at', number)

Pass

In python when statement is required (after semicolon), but we don’t like to execute any code there, we can write the word pass to avoid errors. Also we can use it as a placeholder, for future statements.

Example:

for number in range(6):
    pass

Functions

What is a function? Before we start making functions, let us learn what a function is and why we need them?

Defining a Function

A function is a reusable block of code or programming statements designed to perform a certain task. To define or declare a function, Python provides the def keyword. The following is the syntax for defining a function. The function block of code is executed only if the function is called or invoked.

Declaring and Calling a Function

When we make a function, we call it declaring a function. When we start using the it, we call it calling or invoking a function. Functions can be declared with or without parameters.

# syntax
# Declaring a function
def function_name():
    codes
    codes
# Calling a function
function_name()

Function without Parameters

Function can be declared without parameters.

Example:

def generate_full_name ():
    first_name = 'Asabeneh'
    last_name = 'Yetayeh'
    space = ' '
    full_name = first_name + space + last_name
    print(full_name)
generate_full_name () # calling a function
 
def add_two_numbers ():
    num_one = 2
    num_two = 3
    total = num_one + num_two
    print(total)
add_two_numbers()

Function Returning a Value - Part 1

Functions return values using the return statement. If a function has no return statement, it returns None. Let us rewrite the above functions using return. From now on, we get a value from a function when we call the function and print it.

def generate_full_name ():
    first_name = 'Asabeneh'
    last_name = 'Yetayeh'
    space = ' '
    full_name = first_name + space + last_name
    return full_name
print(generate_full_name())
 
def add_two_numbers ():
    num_one = 2
    num_two = 3
    total = num_one + num_two
    return total
print(add_two_numbers())

Function with Parameters

In a function we can pass different data types(number, string, boolean, list, tuple, dictionary or set) as parameters.

Single Parameter: If our function takes a parameter we should call our function with an argument

  # syntax
  # Declaring a function
  def function_name(parameter):
    codes
    codes
  # Calling function
  print(function_name(argument))

Example:

def greetings (name):
    message = name + ', welcome to Python for Everyone!'
    return message
 
print(greetings('Asabeneh'))
 
def add_ten(num):
    ten = 10
    return num + ten
print(add_ten(90))
 
def square_number(x):
    return x * x
print(square_number(2))
 
def area_of_circle (r):
    PI = 3.14
    area = PI * r ** 2
    return area
print(area_of_circle(10))
 
def sum_of_numbers(n):
    total = 0
    for i in range(n+1):
        total+=i
    return total
print(sum_of_numbers(10)) # 55
print(sum_of_numbers(100)) # 5050

Two Parameter: A function may or may not have a parameter or parameters. A function may also have two or more parameters. If our function takes parameters we should call it with arguments. Let us check a function with two parameters:

  # syntax
  # Declaring a function
  def function_name(para1, para2):
    codes
    codes
  # Calling function
  print(function_name(arg1, arg2))

Example:

def generate_full_name (first_name, last_name):
    space = ' '
      full_name = first_name + space + last_name
      return full_name
print('Full Name: ', generate_full_name('Asabeneh','Yetayeh'))
 
def sum_two_numbers (num_one, num_two):
    sum = num_one + num_two
    return sum
print('Sum of two numbers: ', sum_two_numbers(1, 9))
 
def calculate_age (current_year, birth_year):
    age = current_year - birth_year
    return age 
 
print('Age: ', calculate_age(2021, 1819))
 
def weight_of_object (mass, gravity):
    weight = str(mass * gravity)+ ' N' # the value has to be changed to a string first
    return weight
print('Weight of an object in Newtons: ', weight_of_object(100, 9.81))

Passing Arguments with Key and Value

If we pass the arguments with key and value, the order of the arguments does not matter.

# syntax
# Declaring a function
def function_name(para1, para2):
    codes
    codes
# Calling function
print(function_name(para1 = 'John', para2 = 'Doe')) # the order of arguments does not matter here

Example:

def print_fullname(firstname, lastname):
    space = ' '
    full_name = firstname  + space + lastname
    print(full_name)
print_fullname(firstname = 'Asabeneh', lastname = 'Yetayeh')
 
def add_two_numbers (num1, num2):
    total = num1 + num2
    return total
print(add_two_numbers(num2 = 3, num1 = 2)) # Order does not matter

Function Returning a Value - Part 2

If we do not return a value with a function, then our function is returning None by default. To return a value with a function we use the keyword return followed by the variable we are returning. We can return any kind of data types from a function.

Returning a string: Example:

def print_name(firstname):
    return firstname
print_name('Asabeneh') # Asabeneh
 
def print_full_name(firstname, lastname):
    space = ' '
    full_name = firstname  + space + lastname
    return full_name
print_full_name(firstname='Asabeneh', lastname='Yetayeh')

Returning a number:

Example:

def add_two_numbers (num1, num2):
    total = num1 + num2
    return total
print(add_two_numbers(2, 3))
 
def calculate_age (current_year, birth_year):
    age = current_year - birth_year
    return age
print('Age: ', calculate_age(2019, 1819))

Returning a boolean: Example:

def is_even (n):
    if n % 2 == 0:
        return True    # return stops further execution of the function, similar to break 
    return False
print(is_even(10)) # True
print(is_even(7)) # False

Returning a list: Example:

def find_even_numbers(n):
    evens = []
    for i in range(n + 1):
        if i % 2 == 0:
            evens.append(i)
    return evens
print(find_even_numbers(10))

Function with Default Parameters

Sometimes we pass default values to parameters, when we invoke the function. If we do not pass arguments when calling the function, their default values will be used.

# syntax
# Declaring a function
def function_name(param = value):
    codes
    codes
# Calling function
function_name()
function_name(arg)

Example:

def greetings (name = 'Peter'):
    message = name + ', welcome to Python for Everyone!'
    return message
print(greetings())
print(greetings('Asabeneh'))
 
def generate_full_name (first_name = 'Asabeneh', last_name = 'Yetayeh'):
    space = ' '
    full_name = first_name + space + last_name
    return full_name
 
print(generate_full_name())
print(generate_full_name('David','Smith'))
 
def calculate_age (birth_year,current_year = 2021):
    age = current_year - birth_year
    return age 
print('Age: ', calculate_age(1821))
 
def weight_of_object (mass, gravity = 9.81):
    weight = str(mass * gravity)+ ' N' # the value has to be changed to string first
    return weight
print('Weight of an object in Newtons: ', weight_of_object(100)) # 9.81 - average gravity on Earth's surface
print('Weight of an object in Newtons: ', weight_of_object(100, 1.62)) # gravity on the surface of the Moon

Arbitrary Number of Arguments

If we do not know the number of arguments we pass to our function, we can create a function which can take arbitrary number of arguments by adding * before the parameter name.

# syntax
# Declaring a function
def function_name(*args):
    codes
    codes
# Calling function
function_name(param1, param2, param3,..)

Example:

def sum_all_nums(*nums):
    total = 0
    for num in nums:
        total += num     # same as total = total + num 
    return total
print(sum_all_nums(2, 3, 5)) # 10

Default and Arbitrary Number of Parameters in Functions

def generate_groups (team,*args):
    print(team)
    for i in args:
        print(i) 
generate_groups('Team-1','Asabeneh','Brook','David','Eyob')

Dictionary unpacking

You can call a function which has named arguments using a dictionary with matching key names. You do so using **.

# Define a function that takes two arguments: 'name' and 'location'
def greet(name, location):
    # Print a greeting message using the provided arguments
    print("Hi there", name, "how is the weather in", location)
 
# Call the function using keyword arguments
greet(name="Alice", location="New York")  
# Output: Hi there Alice how is the weather in New York
 
# Create a dictionary with keys matching the function's parameter names
my_dict = {"name": "Alice", "location": "New York"}
 
# Call the function using dictionary unpacking
greet(**my_dict)  
# The ** operator unpacks the dictionary, passing its key-value pairs 
# as keyword arguments to the function.
# Output: Hi there Alice how is the weather in New York

Arbitrary Number of Named Arguments

You can also define a function to accept an arbitrary number of named arguments.

def arbitrary_named_args(**args):
    print("I received an arbitrary number of arguments, totaling", len(args))
    print("They are provided as a dictionary in my function:", type(args))
    print("Let's print them:")
    for k, v in args.items():
        print(" * key:", k, "value:", v)

Generally avoid this unless required as it makes it harder to understand what the function accepts and does.

Function as a Parameter of Another Function

#You can pass functions around as parameters
def square_number (n):
    return n ** n
def do_something(f, x):
    return f(x)
print(do_something(square_number, 3)) # 27

Modules

What is a Module

A module is a file containing a set of codes or a set of functions which can be included to an application. A module could be a file containing a single variable, a function or a big code base.

Creating a Module

To create a module we write our codes in a python script and we save it as a .py file. Create a file named mymodule.py inside your project folder. Let us write some code in this file.

# mymodule.py file
def generate_full_name(firstname, lastname):
    return firstname + ' ' + lastname

Create main.py file in your project directory and import the mymodule.py file.

Importing a Module

To import the file we use the import keyword and the name of the file only.

# main.py file
import mymodule
print(mymodule.generate_full_name('Asabeneh', 'Yetayeh')) # Asabeneh Yetayeh

Import Functions from a Module

We can have many functions in a file and we can import all the functions differently.

# main.py file
from mymodule import generate_full_name, sum_two_nums, person, gravity
print(generate_full_name('Asabneh','Yetayeh'))
print(sum_two_nums(1,9))
mass = 100
weight = mass * gravity
print(weight)
print(person['firstname'])

Import Functions from a Module and Renaming

During importing we can rename the name of the module.

# main.py file
from mymodule import generate_full_name as fullname, sum_two_nums as total, person as p, gravity as g
print(fullname('Asabneh','Yetayeh'))
print(total(1, 9))
mass = 100 
weight = mass * g
print(weight)
print(p)
print(p['firstname'])

Import Built-in Modules

Like other programming languages we can also import modules by importing the file/function using the key word import. Let’s import the common module we will use most of the time. Some of the common built-in modules: math, datetime, os,sys, random, statistics, collections, json,re

OS Module

Using python os module it is possible to automatically perform many operating system tasks. The OS module in Python provides functions for creating, changing current working directory, and removing a directory (folder), fetching its contents, changing and identifying the current directory.

# import the module
import os
# Creating a directory
os.mkdir('directory_name')
# Changing the current directory
os.chdir('path')
# Getting current working directory
os.getcwd()
# Removing directory
os.rmdir()

Sys Module

The sys module provides functions and variables used to manipulate different parts of the Python runtime environment. Function sys.argv returns a list of command line arguments passed to a Python script. The item at index 0 in this list is always the name of the script, at index 1 is the argument passed from the command line.

Example of a script.py file:

import sys
#print(sys.argv[0], argv[1],sys.argv[2])  # this line would print out: filename argument1 argument2
print('Welcome {}. Enjoy  {} challenge!'.format(sys.argv[1], sys.argv[2]))

Now to check how this script works I wrote in command line:

python script.py Asabeneh 30DaysOfPython

The result:

Welcome Asabeneh. Enjoy  30DayOfPython challenge!

Some useful sys commands:

# to exit sys
sys.exit()
# To know the largest integer variable it takes
sys.maxsize
# To know environment path
sys.path
# To know the version of python you are using
sys.version

Statistics Module

The statistics module provides functions for mathematical statistics of numeric data. The popular statistical functions which are defined in this module: mean, median, mode, stdev etc.

from statistics import * # importing all the statistics modules
ages = [20, 20, 4, 24, 25, 22, 26, 20, 23, 22, 26]
print(mean(ages))       # ~22.9
print(median(ages))     # 23
print(mode(ages))       # 20
print(stdev(ages))      # ~2.3

Math Module

Module containing many mathematical operations and constants.

import math
print(math.pi)           # 3.141592653589793, pi constant
print(math.sqrt(2))      # 1.4142135623730951, square root
print(math.pow(2, 3))    # 8.0, exponential function
print(math.floor(9.81))  # 9, rounding to the lowest
print(math.ceil(9.81))   # 10, rounding to the highest
print(math.log10(100))   # 2, logarithm with 10 as base

Now, we have imported the math module which contains lots of function which can help us to perform mathematical calculations. To check what functions the module has got, we can use help(math), or dir(math). This will display the available functions in the module. If we want to import only a specific function from the module we import it as follows:

from math import pi
print(pi)

It is also possible to import multiple functions at once

 
from math import pi, sqrt, pow, floor, ceil, log10
print(pi)                 # 3.141592653589793
print(sqrt(2))            # 1.4142135623730951
print(pow(2, 3))          # 8.0
print(floor(9.81))        # 9
print(ceil(9.81))         # 10
print(math.log10(100))    # 2

But if we want to import all the function in math module we can use * .

from math import *
print(pi)                  # 3.141592653589793, pi constant
print(sqrt(2))             # 1.4142135623730951, square root
print(pow(2, 3))           # 8.0, exponential
print(floor(9.81))         # 9, rounding to the lowest
print(ceil(9.81))          # 10, rounding to the highest
print(math.log10(100))     # 2

When we import we can also rename the name of the function.

from math import pi as  PI
print(PI) # 3.141592653589793

String Module

A string module is a useful module for many purposes. The example below shows some use of the string module.

import string
print(string.ascii_letters) # abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
print(string.digits)        # 0123456789
print(string.punctuation)   # !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~

Random Module

By now you are familiar with importing modules. Let us do one more import to get very familiar with it. Let us import random module which gives us a random number between 0 and 0.9999… The random module has lots of functions but in this section we will only use random and randint.

from random import random, randint
print(random())   # it doesn't take any arguments; it returns a value between 0 and 0.9999
print(randint(5, 20)) # it returns a random integer number between [5, 20] inclusive

List Comprehension

List comprehension in Python is a compact way of creating a list from a sequence. It is a short way to create a new list. List comprehension is considerably faster than processing a list using the for loop.

# syntax
[expression for i in iterable if condition]

Example:1

For instance if you want to change a string to a list of characters. You can use a couple of methods. Let’s see some of them:

# One way
language = 'Python'
lst = list(language) # changing the string to list
print(type(lst))     # list
print(lst)           # ['P', 'y', 't', 'h', 'o', 'n']
 
# Second way: list comprehension
lst = [i for i in language]
print(type(lst)) # list
print(lst)       # ['P', 'y', 't', 'h', 'o', 'n']

Example:2

For instance if you want to generate a list of numbers

# Generating numbers
numbers = [i for i in range(11)]  # to generate numbers from 0 to 10
print(numbers)                    # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
 
# It is possible to do mathematical operations during iteration
squares = [i * i for i in range(11)]
print(squares)                    # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
 
# It is also possible to make a list of tuples
numbers = [(i, i * i) for i in range(11)]
print(numbers)                             # [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]

Example:2

List comprehension can be combined with if expression

# Generating even numbers
even_numbers = [i for i in range(21) if i % 2 == 0]  # to generate even numbers list in range 0 to 21
print(even_numbers)                    # [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
 
# Generating odd numbers
odd_numbers = [i for i in range(21) if i % 2 != 0]  # to generate odd numbers in range 0 to 21
print(odd_numbers)                      # [1, 3, 5, 7, 9, 11, 13, 15, 17, 19]
# Filter numbers: let's filter out positive even numbers from the list below
numbers = [-8, -7, -3, -1, 0, 1, 3, 4, 5, 7, 6, 8, 10]
positive_even_numbers = [i for i in numbers if i % 2 == 0 and i > 0]
print(positive_even_numbers)                    # [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
 
# Flattening a two dimensional array
list_of_lists = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened_list = [ number for row in list_of_lists for number in row]
print(flattened_list)    # [1, 2, 3, 4, 5, 6, 7, 8, 9]

Lambda Function

Lambda function is a small anonymous function without a name. It can take any number of arguments, but can only have one expression. Lambda function is similar to anonymous functions in JavaScript. We need it when we want to write an anonymous function inside another function.

Creating a Lambda Function

To create a lambda function we use lambda keyword followed by a parameter(s), followed by an expression. See the syntax and the example below. Lambda function does not use return but it explicitly returns the expression.

# syntax
x = lambda param1, param2, param3: param1 + param2 + param3
print(x(arg1, arg2, arg3))

Example:

# Named function
def add_two_nums(a, b):
    return a + b
 
print(add_two_nums(2, 3))     # 5
# Lets change the above function to a lambda function
add_two_nums = lambda a, b: a + b
print(add_two_nums(2,3))    # 5
 
# Self invoking lambda function
(lambda a, b: a + b)(2,3) # 5 - need to encapsulate it in print() to see the result in the console
 
square = lambda x : x ** 2
print(square(3))    # 9
cube = lambda x : x ** 3
print(cube(3))    # 27
 
# Multiple variables
multiple_variable = lambda a, b, c: a ** 2 - 3 * b + 4 * c
print(multiple_variable(5, 5, 3)) # 22

Lambda Function Inside Another Function

Using a lambda function inside another function.

def power(x):
    return lambda n : x ** n
 
cube = power(2)(3)   # function power now need 2 arguments to run, in separate rounded brackets
print(cube)          # 8
two_power_of_five = power(2)(5) 
print(two_power_of_five)  # 32

Higher Order Functions

In Python functions are treated as first class citizens, allowing you to perform the following operations on functions:

A function can take one or more functions as parameters
A function can be returned as a result of another function
A function can be modified
A function can be assigned to a variable

In this section, we will cover:

Handling functions as parameters
Returning functions as return value from another functions
Using Python closures and decorators

Function as a Parameter

def sum_numbers(nums):  # normal function
    return sum(nums)    # a sad function abusing the built-in sum function :<
 
def higher_order_function(f, lst):  # function as a parameter
    summation = f(lst)
    return summation
result = higher_order_function(sum_numbers, [1, 2, 3, 4, 5])
print(result)       # 15

Function as a Return Value

def square(x):          # a square function
    return x ** 2
 
def cube(x):            # a cube function
    return x ** 3
 
def absolute(x):        # an absolute value function
    if x >= 0:
        return x
    else:
        return -(x)
 
def higher_order_function(type): # a higher order function returning a function
    if type == 'square':
        return square
    elif type == 'cube':
        return cube
    elif type == 'absolute':
        return absolute
 
result = higher_order_function('square')
print(result(3))       # 9
result = higher_order_function('cube')
print(result(3))       # 27
result = higher_order_function('absolute')
print(result(-3))      # 3

You can see from the above example that the higher order function is returning different functions depending on the passed parameter

Python Closures

Python allows a nested function to access the outer scope of the enclosing function. This is is known as a Closure. Let us have a look at how closures work in Python. In Python, closure is created by nesting a function inside another encapsulating function and then returning the inner function. See the example below.

Example:

def add_ten():
    ten = 10
    def add(num):
        return num + ten
    return add
 
closure_result = add_ten()
print(closure_result(5))  # 15
print(closure_result(10))  # 20

Python Decorators

A decorator is a design pattern in Python that allows a user to add new functionality to an existing object without modifying its structure. Decorators are usually called before the definition of a function you want to decorate.

Creating Decorators

To create a decorator function, we need an outer function with an inner wrapper function.

Example:

# Normal function
def greeting():
    return 'Welcome to Python'
def uppercase_decorator(function):
    def wrapper():
        func = function()
        make_uppercase = func.upper()
        return make_uppercase
    return wrapper
g = uppercase_decorator(greeting)
print(g())          # WELCOME TO PYTHON
 
## Let us implement the example above with a decorator
 
'''This decorator function is a higher order function
that takes a function as a parameter'''
def uppercase_decorator(function):
    def wrapper():
        func = function()
        make_uppercase = func.upper()
        return make_uppercase
    return wrapper
@uppercase_decorator
def greeting():
    return 'Welcome to Python'
print(greeting())   # WELCOME TO PYTHON

Applying Multiple Decorators to a Single Function

 
'''These decorator functions are higher order functions
that take functions as parameters'''
 
# First Decorator
def uppercase_decorator(function):
    def wrapper():
        func = function()
        make_uppercase = func.upper()
        return make_uppercase
    return wrapper
 
# Second decorator
def split_string_decorator(function):
    def wrapper():
        func = function()
        splitted_string = func.split()
        return splitted_string
    return wrapper
 
#Decorators will be executed from bottom to top
@split_string_decorator
@uppercase_decorator     # order with decorators is important in this case - .upper() function does not work with lists
def greeting():
    return 'Welcome to Python'
print(greeting())   # ['WELCOME', 'TO', 'PYTHON']

Accepting Parameters in Decorator Functions

Most of the time we need our functions to take parameters, so we might need to define a decorator that accepts parameters.

def decorator_with_parameters(function):
    def wrapper_accepting_parameters(para1, para2, para3):
        function(para1, para2, para3)
        print("I live in {}".format(para3))
    return wrapper_accepting_parameters
 
@decorator_with_parameters
def print_full_name(first_name, last_name, country):
    print("I am {} {}. I love to teach.".format(
        first_name, last_name))
 
print_full_name("Asabeneh", "Yetayeh",'Finland')

Built-in Higher Order Functions

Some of the built-in higher order functions that we cover in this part are map(), filter, and reduce. Lambda function can be passed as a parameter and the best use case of lambda functions is in functions like map, filter and reduce.

Python - Map Function

The map() function is a built-in function that takes a function and iterable as parameters.

    # syntax
    map(function, iterable)

Example:1

numbers = [1, 2, 3, 4, 5] # iterable
def square(x):
    return x ** 2
numbers_squared = map(square, numbers)
print(list(numbers_squared))    # [1, 4, 9, 16, 25]
# Lets apply it with a lambda function
numbers_squared = map(lambda x : x ** 2, numbers)
print(list(numbers_squared))    # [1, 4, 9, 16, 25]

Example:2

numbers_str = ['1', '2', '3', '4', '5']  # iterable
numbers_int = map(int, numbers_str)
print(list(numbers_int))    # [1, 2, 3, 4, 5]

Example:3

names = ['Asabeneh', 'Lidiya', 'Ermias', 'Abraham']  # iterable
 
def change_to_upper(name):
    return name.upper()
 
names_upper_cased = map(change_to_upper, names)
print(list(names_upper_cased))    # ['ASABENEH', 'LIDIYA', 'ERMIAS', 'ABRAHAM']
 
# Let us apply it with a lambda function
names_upper_cased = map(lambda name: name.upper(), names)
print(list(names_upper_cased))    # ['ASABENEH', 'LIDIYA', 'ERMIAS', 'ABRAHAM']

What actually map does is iterating over a list. For instance, it changes the names to upper case and returns a new list.

Python - Filter Function

The filter() function calls the specified function which returns boolean for each item of the specified iterable (list). It filters the items that satisfy the filtering criteria.

    # syntax
    filter(function, iterable)

Example:1

# Lets filter only even nubers
numbers = [1, 2, 3, 4, 5]  # iterable
 
def is_even(num):
    if num % 2 == 0:
        return True
    return False
 
even_numbers = filter(is_even, numbers)
print(list(even_numbers))       # [2, 4]

Example:2

numbers = [1, 2, 3, 4, 5]  # iterable
 
def is_odd(num):
    if num % 2 != 0:
        return True
    return False
 
odd_numbers = filter(is_odd, numbers)
print(list(odd_numbers))       # [1, 3, 5]

# Filter long name
names = ['Asabeneh', 'Lidiya', 'Ermias', 'Abraham']  # iterable
def is_name_long(name):
    if len(name) > 7:
        return True
    return False
 
long_names = filter(is_name_long, names)
print(list(long_names))         # ['Asabeneh']

Python - Reduce Function

The reduce() function is defined in the functools module and we should import it from this module. Like map and filter it takes two parameters, a function and an iterable. However, it does not return another iterable, instead it returns a single value. Example:1

numbers_str = ['1', '2', '3', '4', '5']  # iterable
def add_two_nums(x, y):
    return int(x) + int(y)
 
total = reduce(add_two_nums, numbers_str)
print(total)    # 15

Python Error Types

When we write code it is common that we make a typo or some other common error. If our code fails to run, the Python interpreter will display a message, containing feedback with information on where the problem occurs and the type of an error. It will also sometimes gives us suggestions on a possible fix. Understanding different types of errors in programming languages will help us to debug our code quickly and also it makes us better at what we do.

Let us see the most common error types one by one. First let us open our Python interactive shell. Go to your you computer terminal and write ‘python’. The python interactive shell will be opened.

SyntaxError

Example 1: SyntaxError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'hello world'
  File "<stdin>", line 1
    print 'hello world'
                      ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('hello world')?
>>>

As you can see we made a syntax error because we forgot to enclose the string with parenthesis and Python already suggests the solution. Let us fix it.

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print 'hello world'
  File "<stdin>", line 1
    print 'hello world'
                      ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('hello world')?
>>> print('hello world')
hello world
>>>

The error was a SyntaxError. After the fix our code was executed without a hitch. Let see more error types.

NameError

Example 1: NameError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print(age)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'age' is not defined
>>>

As you can see from the message above, name age is not defined. Yes, it is true that we did not define an age variable but we were trying to print it out as if we had had declared it. Now, lets fix this by declaring it and assigning with a value.

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> print(age)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'age' is not defined
>>> age = 25
>>> print(age)
25
>>>

The type of error was a NameError. We debugged the error by defining the variable name.

IndexError

Example 1: IndexError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> numbers = [1, 2, 3, 4, 5]
>>> numbers[5]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>>

In the example above, Python raised an IndexError, because the list has only indexes from 0 to 4 , so it was out of range.

ModuleNotFoundError

Example 1: ModuleNotFoundError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import maths
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'maths'
>>>

In the example above, I added an extra s to math deliberately and ModuleNotFoundError was raised. Lets fix it by removing the extra s from math.

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import maths
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'maths'
>>> import math
>>>

We fixed it, so let’s use some of the functions from the math module.

AttributeError

Example 1: AttributeError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import maths
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'maths'
>>> import math
>>> math.PI
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'math' has no attribute 'PI'
>>>

As you can see, I made a mistake again! Instead of pi, I tried to call a PI constant from maths module. It raised an attribute error, it means, that the attribute does not exist in the module. Lets fix it by changing from PI to pi.

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import maths
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'maths'
>>> import math
>>> math.PI
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'math' has no attribute 'PI'
>>> math.pi
3.141592653589793
>>>

Now, when we call pi from the math module we got the result.

KeyError

Example 1: KeyError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> users = {'name':'Asab', 'age':250, 'country':'Finland'}
>>> users['name']
'Asab'
>>> users['county']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'county'
>>>

As you can see, there was a typo in the key used to get the dictionary value. so, this is a key error and the fix is quite straight forward. Let’s do this!

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> user = {'name':'Asab', 'age':250, 'country':'Finland'}
>>> user['name']
'Asab'
>>> user['county']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'county'
>>> user['country']
'Finland'
>>>

We debugged the error, our code ran and we got the value.

TypeError

Example 1: TypeError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 4 + '3'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'
>>>

In the example above, a TypeError is raised because we cannot add a number to a string. First solution would be to convert the string to int or float. Another solution would be converting the number to a string (the result then would be ‘43’). Let us follow the first fix.

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 4 + '3'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'
>>> 4 + int('3')
7
>>> 4 + float('3')
7.0
>>>

Error removed and we got the result we expected.

ImportError

Example 1: TypeError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from math import power
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'power' from 'math'
>>>

There is no function called power in the math module, it goes with a different name: pow. Let’s correct it:

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from math import power
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'power' from 'math'
>>> from math import pow
>>> pow(2,3)
8.0
>>>

ValueError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> int('12a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: '12a'
>>>

In this case we cannot change the given string to a number, because of the ‘a’ letter in it.

ZeroDivisionError

asabeneh@Asabeneh:~$ python
Python 3.9.6 (default, Jun 28 2021, 15:26:21)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 1/0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero
>>>

We cannot divide a number by zero.

We have covered some of the python error types, if you want to check more about it check the python documentation about python error types. If you are good at reading the error types, then you will be able to fix your bugs fast and you will also become a better programmer.

Exception Handling

Python uses try and except to handle errors gracefully. A graceful exit (or graceful handling) of errors is a simple programming idiom - a program detects a serious error condition and “exits gracefully”, in a controlled manner as a result. Often the program prints a descriptive error message to a terminal or log as part of the graceful exit, this makes our application more robust. The cause of an exception is often external to the program itself. An example of exceptions could be an incorrect input, wrong file name, unable to find a file, a malfunctioning IO device. Graceful handling of errors prevents our applications from crashing.

We have covered the different Python error types in the previous section. If we use try and except in our program, then it will not raise errors in those blocks.

Backend MOC

2026-09-02T05:29:25.019Z

🗺️ Backend Map of Content

Welcome to the syllabus and index for Backend.

📄 Notes

DSA

2026-09-02T05:29:25.019Z

DSA Revision Cheat Sheet

1. Time Complexity

Common Complexities

Complexity	Typical Algorithm
O(1)	Hash lookup
O(log n)	Binary Search
O(n)	Single traversal
O(n log n)	Sorting, Heap
O(n²)	Two nested loops
O(n³)	Floyd Warshall
O(2ⁿ)	Subset Generation
O(n!)	Permutations

Input Size Guide

n	Expected Complexity
≤ 10	O(n!)
≤ 20	O(2ⁿ)
≤ 500	O(n³)
≤ 5000	O(n²)
≤ 10⁶	O(n log n) / O(n)

2. Arrays

Sliding Window

Use when:

Subarray
Contiguous segment
Longest/Shortest window

Template:

int l = 0;
for(int r = 0; r < n; r++) {
    // add nums[r]
 
    while(condition) {
        // remove nums[l]
        l++;
    }
}

Prefix Sum

prefix[i] = prefix[i-1] + arr[i];
sum(l,r) = prefix[r] - prefix[l-1];

Kadane’s Algorithm

Maximum Subarray Sum

int best = 0;
int cur = 0;
 
for(int x : arr){
    cur = max(x, cur + x);
    best = max(best, cur);
}

Complexity: O(n)

3. Sorting

sort(v.begin(), v.end());

Descending:

sort(v.rbegin(), v.rend());

Custom Comparator:

sort(v.begin(), v.end(),
[](auto a, auto b){
    return a.second < b.second;
});

Complexity: O(n log n)

4. Binary Search

Search Space Pattern

while(l <= r){
    int mid = l + (r-l)/2;
 
    if(condition(mid))
        r = mid - 1;
    else
        l = mid + 1;
}

Recognize:

Minimize maximum
Maximize minimum
Search answer

Complexity: O(log n)

5. Hashing

Unordered Map

unordered_map<int,int> mp;
mp[x]++;

Unordered Set

unordered_set<int> st;

Average Complexity:

Insert O(1)
Search O(1)

6. Linked List

Must Know:

Reverse Linked List

ListNode* prev = NULL;
 
while(head){
    ListNode* nxt = head->next;
    head->next = prev;
    prev = head;
    head = nxt;
}

Fast Slow Pointer

Applications:

Middle Node
Cycle Detection

7. Stack

Applications:

Monotonic Stack

Used For:

Next Greater Element
Largest Rectangle Histogram

stack<int> st;

Complexity: O(n)

8. Queue / Deque

BFS

queue<int> q;
q.push(src);

Sliding Window Maximum

deque<int> dq;

Complexity: O(n)

9. Heap

Max Heap

priority_queue<int> pq;

Min Heap

priority_queue<int,
vector<int>,
greater<int>> pq;

Applications:

K Largest
K Smallest
Dijkstra

Complexity:

Push O(log n)
Pop O(log n)

10. Trees

DFS

void dfs(int node){
    for(auto child : adj[node])
        dfs(child);
}

BFS

queue<int> q;

Must Know:

Height
Diameter
LCA concept
BST properties

11. Graphs

Representation

vector<int> adj[n];

DFS

dfs(node);

BFS

queue<int> q;

Applications:

Connected Components
Shortest Path (Unweighted)
Cycle Detection

12. Topological Sort

Kahn Algorithm

queue<int> q;

Condition:

Directed Acyclic Graph

Complexity:
O(V + E)

13. Union Find (DSU)

int find(int x){
    if(parent[x]==x)
        return x;
    return parent[x]=find(parent[x]);
}

Operations:

Find
Union

Complexity:
Almost O(1)

Applications:

Kruskal
Connected Components

14. Dijkstra

For positive weights

priority_queue<
pair<int,int>,
vector<pair<int,int>>,
greater<pair<int,int>>
> pq;

Complexity:

O((V+E) log V)

15. Dynamic Programming

DP Thinking

State
Transition
Base Case
Memoization / Tabulation

Fibonacci

dp[i] = dp[i-1] + dp[i-2];

Knapsack

dp[i][w]

LIS

O(n log n) preferred

House Robber

Classic 1D DP

16. Backtracking

Template

void solve(){
    if(base){
        ans.push_back(path);
        return;
    }
 
    for(choice){
        choose;
        solve();
        unchoose;
    }
}

Applications:

Subsets
Permutations
Combination Sum

17. Bit Manipulation

Set Bit

n |= (1 << i);

Clear Bit

n &= ~(1 << i);

Toggle Bit

n ^= (1 << i);

Check Bit

(n >> i) & 1

Count Bits

__builtin_popcount(n);

18. Math

GCD

gcd(a,b)

Complexity:
O(log n)

LCM

a / gcd(a,b) * b

Fast Power

while(n){
    if(n&1) ans*=x;
    x*=x;
    n>>=1;
}

Complexity:
O(log n)

Modular Arithmetic

const int MOD = 1e9+7;

Rules:

(a+b)%MOD
(a*b)%MOD
(a-b+MOD)%MOD

19. STL You Must Know

vector
pair
set
unordered_set
map
unordered_map
stack
queue
deque
priority_queue
sort()
lower_bound()
upper_bound()

20. Placement Checklist

Arrays & Strings ✓
Binary Search ✓
Hashing ✓
Linked List ✓
Stack ✓
Queue ✓
Heap ✓
Trees ✓
Graphs BFS/DFS ✓
Topological Sort ✓
DSU ✓
Dijkstra ✓
Dynamic Programming ✓
Backtracking ✓
Bit Manipulation ✓
Math + GCD ✓

If you can solve medium-level questions from all the above topics, you are ready for most college placement coding rounds.

Dsa MOC

2026-09-02T05:29:25.019Z

🗺️ Dsa Map of Content

Welcome to the syllabus and index for Dsa.

📄 Notes

Ansible

2026-09-02T05:29:25.019Z

Ansible is an open source, command-line IT automation software application written in Python. It can configure systems, deploy software, and orchestrate advanced workflows to support application deployment, system updates, and more. Ansible provides open-source automation that reduces complexity and runs everywhere. Using Ansible lets you automate virtually any task. Here are some common use cases for Ansible:

Eliminate repetition and simplify workflows
Manage and maintain system configuration
Continuously deploy complex software
Perform zero-downtime rolling updates

Control node

A system on which Ansible is installed. You run Ansible commands such as ansible or ansible-inventory on a control node.

Inventory

A list of managed nodes that are logically organized. You create an inventory on the control node to describe host deployments to Ansible.

Managed node

A remote system, or host, that Ansible controls.

Ansible uses playbooks to automate tasks, you describe the desired state of the system in the playbook and ansible makes sure that the system remains in that state.

→ As automation technology, Ansible is designed around the following principles:

1. Agent-less architecture

Low maintenance overhead by avoiding the installation of additional software across IT infrastructure.

2. Simplicity

Automation playbooks use straightforward YAML syntax for code that reads like documentation. Ansible is also decentralized, using SSH with existing OS credentials to access remote machines.

3. Scalability and flexibility

Easily and quickly scale the systems you automate through a modular design that supports a large range of operating systems, cloud platforms, and network devices.

4. Idempotence and predictability

When the system is in the state your playbook describes, Ansible does not change anything, even if the playbook runs multiple times.

Docker

2026-09-02T05:29:25.019Z

Introduction

The issue: It works on my machine
The solution: Containerize it
Basically Docker is a tool which helps us to make containers, and these containers run applications inside them.
Docker makes a OS independent environment and makes the system compatible for the libraries that are required

How did Docker came?

There was a company names DotCloud which started facing the classic problem of “it works on my machine”, so they decided to make a tool for building containers and they made it available in pycon in 2013 and in 2017 it became a CNCF tool.

Virtualization vs Containerization

Virtual Machines are used.	Containerization tool like Docker, Podman, Containerd are used.
uses Hypervisor: allocates resources from the system and runs the os on it. Vistual machines has there own operating system.	Docker engine uses the host operating system, do not require there own operating system. The system calls are made via the hosto operating system only.
Virtualization is heavy.	Containers are light weight.

Docker Architecture

Docker Engine: Application container engine, which runs all the applications.
Docker Daemon: Background service to manage all the running applications. Dockerd runs containerd internally
Docker CLI: Interface to interact.
Docker Client: Connects APIs and communicates to docker engine and gives as a GUI view of our containers.

Docker Images and Containers

docker ps (to see the running continers/processes)
docker images (to see the available docker images)

A Docker image is a read-only blueprint that contains the source code, libraries, dependencies, and runtime required to run an application, while a Docker container is the live, runnable instance created from that image. A docker image is written using a Dockerfile.

Docker file has instructions → these instructions create image → this image can be replicated in the form of containers.

Dockerfile → image → container (build) (run)

docker pull hello-world

The docker pull command is used to pull a pre-defined image from the docker hub.

docker run hello-world

This command will run the container for the image.

Adding env to the docker containers

docker run -e MYSQL_ROOT_PASSWORD=root mysql

“Use -d to run the container in the background”

docker stop {container id}
docker start {container id}

How to write a Dockerfile?

FROM {base_image_name:image_version}           #pulls base image

WORKDIR /app                                   #directory in the container                                                       where the application code will                                                  reside

COPY . .                                       #copy code from the current                                                       location to container's workdir

RUN apt-get update && apt-get install -y curl  #executes commands at image                                                       build time

CMD ["python","app.py"]                        #specifies the default command                                                    executed at container runtime

ENTRYPOINT                                     #similar to CMD but it cannot be                                                  overridden

CMD can be override at the time of docker run command, but ENTRYPOINT cannot be overridden, both of these commands are used to inject commands in the running containers.

How to make a docker image from a Dockerfile?

docker build -t test-app .

-t → gives a tag or name to the docker image . → give the context of the Dockerfile (location of the Dockerfile to be used)

docker run test-app

This will run the docker image as a container
Everytime there is a change in the source code the Docker image should be updates using “docker build” command.

→ Port mapping

docker run -p 80:80 test-app

Host Port : Container Port
-p : publish

→ How to get inside a docker container

docker exec -it {container id} bash

-it : interactive terminal
bash : shell name

Docker Networking

Imagine your computer runs multiple docker containers and both the containers are isolated in nature, but if we want both of these containers to communicate we need a network for that.

→ Types of Docker networks

Host (docker m/c and host m/c ‘s networks are same)
Bridge (bridges the network between host and the container, it is the default network for docker)
User defined Bridge (custom bridge)
None (completely isolated container)
MACVLAN (network in which communication happens via 2 MAC addresses, used in docker swarm)
IPVLAN (used in docker swarm)
Overlay (used in docker swarm)

docker network ls

Shows all the available docker networks.

docker network create mynetwork -d bridge

-d : drivers
creates a bridge type network

→ How to give network to the containers?

	docker run -d --name mysql --network two-tier -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=devops mysql:latest

To get information about the network:

docker network inspect two-tier

Docker Volumes and Storage

When we delete our docker containers the data which was not persistent in nature will we erased. For saving our data after a crash we bind our data with our host machine which are called volumes.

docker volume ls

Shows all the docker volumes that are currently available or are in existence.

docker volume create mysql-data

A docker volume can be inspected using the “docker inspect {volume name}”
binding of the container with the host path of the volume:

docker run -d --name mysql --network two-tier -v mysql-data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=devops mysql:latest

Docker Compose

Docker compose is a config file that is a yaml/yml file.

key: value We can make multiple docker containers in a docker compose file.

version: "3.8"

services:
  mysql:
    container_name: mysql
    image: mysql
    environment:
      MYSQL_DATABASE: "devops"
      MYSQL_ROOT_PASSWORD: "root"
    ports:
      - "3306:3306"
    volumes:
      - mysql-data:/var/lib/mysql
    networks:
      - two-tier
    restart: always #no, unless-stopped, on-failure
    healthcheck:
      test: ["CMD", "mysqladmin","ping","-h","localhost","-uroot","-proot"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 60s
  flask:
    build:
      context: .
    container_name: two-tier-backend
    ports:
      - "5000:5000"
    environment:
      MYSQL_HOST: mysql
      MYSQL_USER: root
      MYSQL_PASSWORD: root
	  MYSQL_DB: devops
	networks:
	  - two-tier
	depends_on:
	  - mysql
volumes:
  - mysql-data
networks:
  - two-tier

version: syntax or latest version of docker compose.
Inside services block we mention the names of containers we want to build.
Flask or the other backend images are not made they are build using the context (location of the docker file).
depends_on: This keyword says that the container is dependent on the other container, so the other container should be made first.
healthcheck block: This block makes sure that the container is fully up and working, until this health check is not passed the container is not considered full up.
restart: Docker Compose supports four core restart policies to control how containers behave when they stop, crash, or when the system reboots. Restart policies work as per the results of the healthckecks.
- "no": The default behavior. Docker will never automatically restart the container.
- always: Always restarts the container regardless of why it stopped or its exit code.
- unless-stopped: Similar to always, but respects manual intervention. If you intentionally run docker compose stop, the container will not restart automatically on system or daemon reboots.
- on-failure: Restarts the container only if it exits with a non-zero exit code (indicating an application error or crash). You can optionally specify maximum attempts using on-failure:5

docker compose up

This command is used to build and run the docker compose file.

docker compose up -d

This command is used to build and run docker compose in the background.

docker compose down

Stops the docker containers made using docker compose.

docker compose up -d --build

Forcefully pulls and builds all the docker images again.

docker system prune

Deletes all the unused or stopped containers.

 docker attach {image_id}

attached the containers or container logs to the screen.

Docker Registry

A centralized storage and distribution system used to manage and share Docker images.

Step 1: Docker login Login to docker using the cli.
```
docker login
```
Step 2: Tag the image

docker image tag {old_name_of_the_image}:{version_of_the_image}{docker_username}/{new_image_name}:{version}

Step 3: Docker push Pushes the docker image to the dockerhub
```
docker push {image_tagged_name}:{version}
```
Now we do not need to build the image using the build context in the docker compose we can directly use the “image:” and give it the name of our pushed image.
In windows the Docker desktop holds a simulation of linux, which means that if we want to run a linux based docker image on windows we need to install docker desktop on top priority.

Multi-stage Docker Builds

“Docker image optimization” When we use base images like python:3.7, which are big images which turn out the container to be also big which is a concerning thing in terms of optimization as it takes more time in everything (container building, etc).

#stage 1: base image approx of 994 mb ~ 1.40 gb (total size of image)
FROM python:3.7 AS builder

WORKDIR: /app

COPY requirements.txt .

RUN pip install -r requirements.txt

#stage 2 small base image around 400 mb
FROM python:3.7-slim

WORKDIR /app

COPY --from=builder /usr/local/lib/python3.7/site-packages /usr/local/lib/python3.7/site-packages

COPY . .
ENTRYPOINT ["python","run.py"]

Monitoring and logging in Docker

docker logs {container_id}

Used to see the logs of a particular container.

nohup docker attach {container_id} &

nohup runs a command in the background and sends the data or logs to a file.
It will make a nohup.out named file which will store all the data or logs of the particular contianer.

docker scout and docker init

Base image → new image
The base image may contain some security related issues or vulnerabilities so we need to scan the image for which we use docker scout
docker scout is by default installed in the docker desktop but we need to use plugins to install docker scout in linux.
docker init is a command-line utility provided by Docker Desktop that automatically initializes and generates the necessary configuration boilerplate files to containerize your application.

Index for backend

2026-09-02T05:29:24.996Z

DBMS, Advance DB concepts,APIs and its types,Auth,Building for Scale,Django, FastAPI,Postgres,Python,RabbitMQ,Redis,Search Engines,Testing,Web Security

/Private / md-notes

System Design

Introduction

What is System Design?

1. Scalability

1.1 Types of Scaling

However, there are many other factors to consider when determining the appropriate approach. So lets explain one by one.

1.1.1 Vertical Scaling

1.1.2 Horizontal Scaling

2. Caching

2.1 Caching: Mechanisms, Tools, and Techniques

2.2 Cache Strategies: Types and When to Use What

Choosing the Right Caching Strategy

3. Load Balancing

3.1 Types of Load Balancers based on the OSI Model

3.2 Load Balancing Algorithms

3. Database Replication and Partitioning

3.1 Database Replication

3.2 Algorithms to implement Database replication

Single Leader Replication(Active-Passive or Master-Slave Replication)

System Design MOC

🗺️ System Design Map of Content

📄 Notes

Md Notes MOC

🗺️ Md Notes Map of Content

📁 Sub-Topics

📄 Notes

key

Index for Devops

Kubernetes

Kubernetes (often abbreviated as K8s) is an open-source platform that automates deploying, scaling, and managing containerized applications.

1. Kubernetes Architecture

2. Kubernetes works in cluster

3. How to Write Yet Another Markup Language(yml/yaml)

Heading towards practical part…

Creating a kind cluster

- Kubernetes Namespaces

|Docker container→Pod→Deployment→service |

- Kubernetes Pods

- Kubernetes workloads

Networking for Devops

1. How DNS actually works?

1.1 DNS Records

2. How HTTP works?

Nginx

How to write Nginx config file

Config file structure

Serving static content

Nginx Reverse Proxy for an Application Running on localhost:8000

Objective

Architecture

Nginx Configuration

Explanation

1. Listen on Port 80

2. Forward Requests to Backend

3. Preserve Original Request Information

Host Header

Real Client IP

Forwarded Client Chain

Original Protocol

Validate Configuration

Reload Nginx

Verify Backend Service

Test the Reverse Proxy

Troubleshooting

Check Nginx Error Logs

Check Access Logs

Key Concept

Service Mesh

What exactly a service mesh is?

1. Why Do You Need a Service Mesh?

Real-World Example: Netflix

2. How Service Mesh Helps

3. Service Mesh Architecture

3.1 Data Plane (Sidecar Proxies)

3.2 Control Plane

How It Works: Without vs. With Service Mesh

4. Popular Service Mesh Implementations

5. Benefits of Service Mesh

5.1 Security: Encrypted & Secure Communication (mTLS)