Backward Elimination Technique in Multiple Linear Regression

Last updated: February 8, 2026Reviewed date: February 8, 2026Reading time: 6 min read
Patient Tools

Read, save, and share this guide

Use these quick tools to make this medical article easier to read, print, save, or share with a family member.

Patient Mode

Understand this article easily

Switch between simple English and easy Bangla patient notes. This is for education and does not replace a doctor consultation.

As artificial intelligence and machine learning become more prevalent, so too does the need for efficient and accurate methods of training models. Backward elimination is one such method that is commonly used in machine learning. This technique is helpful because it can help to reduce the chances of overfitting the...

For severe symptoms, danger signs, pregnancy, child illness, or sudden worsening, seek urgent medical care.

বাংলা রোগী নোট এখনো যোগ করা হয়নি। পোস্ট এডিটরে “RX Bangla Patient Mode” বক্স থেকে সহজ বাংলা সারাংশ যোগ করুন।

এই তথ্য শিক্ষা ও সচেতনতার জন্য। এটি ডাক্তারি পরীক্ষা, রোগ নির্ণয় বা প্রেসক্রিপশনের বিকল্প নয়।

Article Summary

As artificial intelligence and machine learning become more prevalent, so too does the need for efficient and accurate methods of training models. Backward elimination is one such method that is commonly used in machine learning. This technique is helpful because it can help to reduce the chances of overfitting the data and make the linear regression model more interpretable. The backward elimination technique is used in machine learning to find the best...

Key Takeaways

  • This article explains What Is Backward Elimination Technique in Multiple Linear Regression? in simple medical language.
  • This article explains Forward Selection vs. Backward Elimination in simple medical language.
  • This article explains How to Implement Backward Elimination With Examples in simple medical language.
  • This article explains Our Learners Also Ask in simple medical language.
Educational health guideWritten for patient understanding and clinical awareness.
Reviewed content workflowUse writer and reviewer profiles for stronger trust.
Emergency safety firstUrgent warning signs are highlighted below.

Seek urgent medical care if you notice

These warning signs are general safety guidance. Local emergency numbers and clinical judgment should always come first.

  • New or worsening weakness, numbness, or loss of coordination.
  • Loss of bladder or bowel control, or numbness around the groin or saddle area.
  • Back or neck pain with fever, recent major injury, cancer history, or unexplained weight loss.
1

Emergency now

Use emergency care for severe, sudden, rapidly worsening, or life-threatening symptoms.

2

See a doctor

Book a professional medical evaluation if symptoms persist, worsen, recur often, affect daily activities, or occur in a high-risk patient.

3

Learn safely

Use this article to understand possible causes, tests, treatment options, prevention, and questions to ask your clinician.

Before reading

RX Patient Tools

Use these quick guides before reading the article, or return to them when you need help preparing questions for a doctor.

Start here Choose the right pathway for symptoms, reports, medicines, or urgent warning signs. Disease article roadmap Read this topic step by step: meaning, symptoms, warning signs, diagnosis, treatment, prevention, and follow-up. Treatment planner Prepare questions about treatment choices, benefits, risks, side effects, and follow-up. Family & caregiver guide Organize symptoms, reports, medicines, questions, and follow-up safely. Nutrition & diet guide Prepare food, hydration, supplement, and medicine-timing questions safely. Prevention guide Organize risk factors, protective habits, screening, and warning signs. Recovery guide Prepare a safe plan for activity, rehabilitation, warning signs, and follow-up.
Definition

As artificial intelligence and machine learning become more prevalent, so too does the need for efficient and accurate methods of training models. Backward elimination is one such method that is commonly used in machine learning. This technique is helpful because it can help to reduce the chances of overfitting the data and make the linear regression model more interpretable.

The backward elimination technique is used in machine learning to find the best subset of features from a given set of features. It works by iteratively removing features that are not predictive of the target variable or have the least predictive power. This article will explore the backward elimination technique and how it can be used to train machine learning models and their implementation.

What Is Backward Elimination Technique in Multiple Linear Regression?

Multiple linear Regression is a standard statistical method used to assess the relationships between a dependent variable and a set of independent variables. In many cases, there are too many independent variables to include all of them in the regression model. In these situations, modelers can use a backward elimination process to iteratively remove the least important variables until only the most important ones remain.

Backward elimination is a simple and effective way to select a subset of variables for a linear regression model. It is easy to implement and can be automated. The backward elimination process begins by fitting a multiple linear regression model with all the independent variables. The variable with the highest p-value is removed from the model, and a new model fits. This process is repeated until all variables in the model have a p-value below some threshold, typically 0.05.

Forward Selection vs. Backward Elimination

In machine learning, there are two main methods for feature selection: forward selection and backward elimination. Both methods have pros and cons, and which one you use will ultimately depend on your specific data and goals.

Forward selection is a greedy algorithm that starts with an empty set of features and adds features one by one until the model performance reaches a peak. This method is simple and easy to implement but can be computationally expensive and may not find the optimal set of features.

Backward elimination is a more systematic approach that starts with a complete set of features and removes features one by one until the model performance reaches a peak. This method is more computationally efficient but may not find the optimal set of features either.

So which method should you use? Ultimately, it depends on your specific data and goals. If you have a large set of potential features and want to be more efficient with your feature selection, backward elimination would be a better approach.

How to Implement Backward Elimination With Examples

Backward elimination is a machine learning algorithm that helps you choose the essential features of your data. This algorithm gradually removes features that are not important until only the most essential features remain.

There are many ways to implement backward elimination, but one of the most common methods is to use a p-value threshold. P-values are a measure of how likely a feature is to be necessary. A p-value threshold is a p-value below which a feature will be removed.

For example, let’s say you have a dataset with ten features. You decide to use a p-value threshold of 0.05. Any feature with a p-value greater than 0.05 will be removed.

To implement backward elimination, you first need to calculate the p-values of each feature. Then, you compare the p-values to your threshold and remove the features with p-values greater than the threshold.

You can continue to do this until all of the features have been removed, or you reach your desired number of features.

Let’s take a look at an example. Say you have the following dataset:

Feature feature_1 feature_2 feature_3 feature_4 feature_5
p-value 0.01 0.03 0.05 0.07 0.09

In this dataset, we would remove feature_1 and feature_2 since they have p-values less than our threshold of 0.05. Feature_3 would be kept since its p-value is more significant than our threshold. This process will continue until all of the features have been removed or we reach our desired number of features.

Our Learners Also Ask

1. What is backward elimination in Regression?

Backward elimination is a method used in regression analysis to select a subset of explanatory variables for the model. The model includes the initial and all explanatory variables in backward elimination. Then, the variable with the highest p-value is removed from the model. This process is repeated until all variables in the model have a p-value below a given threshold. Backward elimination is an efficient way to build a regression model with a small number of explanatory variables.

2. What are backward elimination and forward selection?

Backward elimination and forward selection are methods used in feature selection, which is the process of choosing the most relevant features for a model. Backward elimination starts with all features included in the model and then removes the least relevant features one at a time. Forward selection starts with no features included in the model and then adds the most relevant features one at a time.

3. How do you do backward elimination in Python?

Python’s `sklearn` library provides a handy function for backward elimination on a linear regression model. This function is called `backward_elimination()`.

You must first fit a linear regression model to your data to use this function. Then, you can pass the model into the `backward_elimination()` function along with a significance level (usually 0.05). The function will then remove the least significant predictor from the model until all predictors in the model are significant.

4. What are forward and backward Regression?

Forward Regression starts with all the potential predictor variables and removes the ones that are not statistically significant. This is done until only the significant predictors remain. Backward Regression, on the other hand, begins with all the predictor variables and removes the ones that are not statistically significant. This process is repeated until only the significant predictors remain.

5. How do you do a backward elimination in SPSS?

Backward elimination is a statistical method used to find the simplest model that explains the data. In SPSS, backward elimination can be used to find the best model by iteratively removing variables that are not statistically significant.

To do a backward elimination in SPSS, select the variables you want to include in the model. Then, click on the Analyze tab and choose Regression. Next, select the variables you want to remove from the model and click on the Remove button. Finally, click on the Run button to see the model’s results.

Conclusion

The backward elimination technique is a method used in machine learning to improve the accuracy of predictions. This method removes features that are not predictive of the target variable or not statistically significant. Backward elimination is a powerful technique that can improve the accuracy of predictions and help you build better machine-learning models

Doctor visit helper

Prepare before seeing a doctor

A simple rural-patient checklist to help you explain symptoms clearly, ask better questions, and avoid unsafe self-treatment.

Safety note: This is not a prescription or diagnosis. For severe symptoms, pregnancy danger signs, children with serious illness, chest pain, breathing difficulty, stroke-like weakness, or major injury, seek urgent care.

Which doctor may help?

Start with a registered doctor or the nearest qualified health center.

What to tell the doctor

  • Write when the problem started and how it changed.
  • Bring old prescriptions, investigation reports, and current medicines.
  • Write allergies, pregnancy status, diabetes, kidney/liver disease, and major past illnesses.
  • Bring one family member if the patient is weak, elderly, confused, or a child.

Questions to ask

  • What is the most likely cause of my symptoms?
  • Which danger signs mean I should go to hospital quickly?
  • Which tests are necessary now, and which can wait?
  • How should I take medicines safely and what side effects should I watch for?
  • When should I come for follow-up?

Tests to discuss

  • Vital signs: temperature, pulse, blood pressure, oxygen saturation
  • Basic physical examination by a clinician
  • CBC, urine test, blood sugar, or imaging only when clinically needed

Avoid these mistakes

  • Do not use antibiotics, steroid tablets/injections, or strong painkillers without proper medical advice.
  • Do not hide pregnancy, kidney disease, ulcer, allergy, or blood thinner use.
  • Do not delay emergency care when danger signs are present.

Medicine safety and first-aid guide

This section is for patient education only. It does not replace a doctor, pharmacist, or emergency care.

Safe first steps

  • Rest, drink safe water, and observe symptoms carefully.
  • Keep a written note of symptoms, duration, temperature, medicines already taken, and allergy history.
  • Seek medical care quickly if symptoms are severe, worsening, or unusual for the patient.

OTC medicine safety

  • For mild pain or fever, ask a registered pharmacist or doctor before using common over-the-counter pain/fever medicines.
  • Do not combine multiple pain medicines without advice, especially if you have kidney disease, liver disease, stomach ulcer, asthma, pregnancy, or take blood thinners.
  • Do not give adult medicines to children unless a qualified clinician advises it.

Avoid these mistakes

  • Do not start antibiotics without a proper medical decision.
  • Do not use steroid tablets or injections casually for quick relief.
  • Do not delay emergency care because of home remedies.

Get urgent help if

  • Severe symptoms, confusion, fainting, breathing difficulty, chest pain, severe dehydration, or sudden weakness need urgent medical care.
Medicine names, dose, and timing must be decided by a qualified clinician or pharmacist after checking age, pregnancy, allergy, other diseases, and current medicines.

For rural patients and family caregivers

Patient health record and symptom diary

Write your symptoms, medicines already taken, test results, and questions before visiting a doctor. This note stays on your device unless you print or copy it.

Doctor to discuss: Doctor / qualified healthcare provider
Tests to discuss with doctor
  • Basic vital signs: temperature, pulse, blood pressure, oxygen level if needed
  • Relevant blood, urine, imaging, or specialist tests only after clinical assessment
Questions to ask
  • What is the most likely cause of my symptoms?
  • Which warning signs mean I should go to emergency care?
  • Which tests are really needed now?
  • Which medicines are safe for my age, pregnancy status, allergy, kidney/liver/stomach condition, and current medicines?

Emergency warning signs such as chest pain, severe breathing difficulty, sudden weakness, confusion, severe dehydration, major injury, or loss of bladder/bowel control need urgent medical care. Do not wait for online information.

Safe pathway to proper treatment

Care roadmap for: Backward Elimination Technique in Multiple Linear Regression

Use this simple roadmap to understand the next safe steps. It is educational and does not replace examination by a doctor.

Go to emergency care if you notice:
  • Severe or rapidly worsening symptoms
  • Breathing difficulty, chest pain, fainting, confusion, severe weakness, major injury, or severe dehydration
Doctor / service to discuss: Qualified healthcare provider; specialist depends on symptoms and examination.
  1. Step 1

    Check danger signs first

    If danger signs are present, seek emergency care and do not wait for online information.

  2. Step 2

    Record the symptom story

    Write when symptoms started, severity, medicines already taken, allergies, pregnancy status, and test results.

  3. Step 3

    Visit a qualified clinician

    A doctor, nurse, or qualified healthcare provider can examine you and decide which tests or treatment are needed.

  4. Step 4

    Do only useful tests

    Do tests after clinical assessment. Avoid unnecessary tests, random antibiotics, or repeated medicines without diagnosis.

  5. Step 5

    Follow up and return early if worse

    If symptoms worsen, new warning signs appear, or treatment is not helping, return for review quickly.

Rural patient practical tips
  • Take a written symptom diary and all previous prescriptions/test reports.
  • Do not hide medicines already taken, even herbal or over-the-counter medicines.
  • Ask which warning signs mean urgent referral to hospital.

This roadmap is for education. A real diagnosis and treatment plan requires history, examination, and clinical judgment.

RX Patient Help

Ask a health question safely

Write your symptom story. A health professional or site editor can review it before any answer is prepared. This box is not for emergency care.

Emergency first: Severe chest pain, breathing trouble, unconsciousness, stroke signs, severe injury, heavy bleeding, or rapidly worsening symptoms need urgent local medical care now.

Frequently Asked Questions

What Is Backward Elimination Technique in Multiple Linear Regression?

Multiple linear Regression is a standard statistical method used to assess the relationships between a dependent variable and a set of independent variables. In many cases, there are too many independent variables to include all of them in the regression model. In these situations, modelers can use a backward elimination process to iteratively remove the least important variables until only the most important ones remain. Backward elimination is a simple and effective way to select a subset of variables for…

Forward Selection vs. Backward Elimination In machine learning, there are two main methods for feature selection: forward selection and backward elimination. Both methods have pros and cons, and which one you use will ultimately depend on your specific data and goals. Forward selection is a greedy algorithm that starts with an empty set of features and adds features one by one until the model performance reaches a peak. This method is simple and easy to implement but can be computationally expensive and may not find the optimal set of features. Backward elimination is a more systematic approach that starts with a complete set of features and removes features one by one until the model performance reaches a peak. This method is more computationally efficient but may not find the optimal set of features either. So which method should you use? Ultimately, it depends on your specific data and goals. If you have a large set of potential features and want to be more efficient with your feature selection, backward elimination would be a better approach. How to Implement Backward Elimination With Examples Backward elimination is a machine learning algorithm that helps you choose the essential features of your data. This algorithm gradually removes features that are not important until only the most essential features remain. There are many ways to implement backward elimination, but one of the most common methods is to use a p-value threshold. P-values are a measure of how likely a feature is to be necessary. A p-value threshold is a p-value below which a feature will be removed. For example, let's say you have a dataset with ten features. You decide to use a p-value threshold of 0.05. Any feature with a p-value greater than 0.05 will be removed. To implement backward elimination, you first need to calculate the p-values of each feature. Then, you compare the p-values to your threshold and remove the features with p-values greater than the threshold. You can continue to do this until all of the features have been removed, or you reach your desired number of features. Let's take a look at an example. Say you have the following dataset: Feature feature_1 feature_2 feature_3 feature_4 feature_5 p-value 0.01 0.03 0.05 0.07 0.09 In this dataset, we would remove feature_1 and feature_2 since they have p-values less than our threshold of 0.05. Feature_3 would be kept since its p-value is more significant than our threshold. This process will continue until all of the features have been removed or we reach our desired number of features. Our Learners Also Ask 1. What is backward elimination in Regression?

Backward elimination is a method used in regression analysis to select a subset of explanatory variables for the model. The model includes the initial and all explanatory variables in backward elimination. Then, the variable with the highest p-value is removed from the model. This process is repeated until all variables in the model have a p-value below a given threshold. Backward elimination is an efficient way to build a regression model with a small number of explanatory variables.

2. What are backward elimination and forward selection?

Backward elimination and forward selection are methods used in feature selection, which is the process of choosing the most relevant features for a model. Backward elimination starts with all features included in the model and then removes the least relevant features one at a time. Forward selection starts with no features included in the model and then adds the most relevant features one at a time.

3. How do you do backward elimination in Python?

Python's `sklearn` library provides a handy function for backward elimination on a linear regression model. This function is called `backward_elimination()`. You must first fit a linear regression model to your data to use this function. Then, you can pass the model into the `backward_elimination()` function along with a significance level (usually 0.05). The function will then remove the least significant predictor from the model until all predictors in the model are significant.

4. What are forward and backward Regression?

Forward Regression starts with all the potential predictor variables and removes the ones that are not statistically significant. This is done until only the significant predictors remain. Backward Regression, on the other hand, begins with all the predictor variables and removes the ones that are not statistically significant. This process is repeated until only the significant predictors remain.

5. How do you do a backward elimination in SPSS?

Backward elimination is a statistical method used to find the simplest model that explains the data. In SPSS, backward elimination can be used to find the best model by iteratively removing variables that are not statistically significant. To do a backward elimination in SPSS, select the variables you want to include in the model. Then, click on the Analyze tab and choose Regression. Next, select the variables you want to remove from the model and click on the Remove button.…

References

Add references, clinical guidelines, textbooks, journal articles, or trusted medical sources here. You can edit this area from the RX Article Professional Blocks panel.

More from this topic
  1. Semantic Chunks for RAG

    DefinitionIn order to abide by the context window of the LLM , we usually break text into smaller parts / pieces which is called chunking. LLMs,...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  2. RAG Evaluation and Meta-Evaluation with GroUSE

    DefinitionThis tutorial introduces GroUSE, a framework for evaluating Retrieval-Augmented Generation (RAG) pipelines, focusing on the final stage: Grounded Question Answering (GQA). It demonstrates how to use...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  3. Deep Evaluation of RAG Systems using deepeval

    DefinitionThis code demonstrates the use of the deepeval library to perform comprehensive evaluations of Retrieval-Augmented Generation (RAG) systems. It covers various evaluation metrics and provides a framework for...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  4. Simple RAG with Llamaindex

    DefinitionThis code implements a basic Retrieval-Augmented Generation (RAG) system for processing and querying PDF document(s). The system uses a pipeline that encodes the documents and creates...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  5. Simple RAG (Retrieval-Augmented Generation) System

    DefinitionThis code implements a basic Retrieval-Augmented Generation (RAG) system for processing and querying PDF documents. The system encodes the document content into a vector store, which...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  6. Simple RAG (Retrieval-Augmented Generation) System for CSV Files

    DefinitionThis code implements a basic Retrieval-Augmented Generation (RAG) system for processing and querying CSV documents. The system encodes the document content into a vector store, which...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  7. Simple RAG (Retrieval-Augmented Generation) System for CSV Files

    DefinitionThis code implements a basic Retrieval-Augmented Generation (RAG) system for processing and querying CSV documents. The system encodes the document content into a vector store, which...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  8. Semantic Chunking for Document Processing

    DefinitionThis code implements a semantic chunking approach for processing and retrieving information from PDF documents, first proposed by Greg Kamradt and subsequently implemented in LangChain. Unlike traditional methods that...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  9. Self-RAG: A Dynamic Approach to Retrieval-Augmented Generation

    DefinitionSelf-RAG is an advanced algorithm that combines the power of retrieval-based and generation-based approaches in natural language processing. It dynamically decides whether to use retrieved information...

    February 15, 2025 • Rx Data Science and Artificial Intelligence
  10. RAG System with Feedback Loop

    DefinitionThis system implements a Retrieval-Augmented Generation (RAG) approach with an integrated feedback loop. It aims to improve the quality and relevance of responses over time by...

    February 15, 2025 • Rx Data Science and Artificial Intelligence