How to Remove Propositions from a String in Python

A proposition is a statement that expresses a judgment or opinion.  For example: “The sky is blue.”  Propositions are declarative sentences that state something about a subject.   Removing these propositions can help simplify text for further processing.

Steps to Remove Propositions

• Identify Propositions : Detect sentences that are propositions.
• Split the Text into Sentences : We’ll split the text into sentences. This can be done using the nltk library, which provides tools for text processing, including sentence tokenization.
• Define Criteria for Propositions : For this example, we’ll assume that any sentence that ends with a period (.) is a proposition. This is a simplistic approach, but it works for our purpose.
• Remove Propositions : We will filter out sentences that meet our criteria for propositions.
• Testing the Function : Let’s test our function with a sample text.

  import re
  
  # Define the regular expression pattern for propositions
  proposition_pattern = r'\b(?:is|are|was|were|seems|appears|looks|feels|thinks|believes|says|claims|states)\b.*?\.'
  
  def remove_propositions(text):
      # Split text into sentences
      sentences = re.split(r'(?<=[.!?]) +', text)
      
      # Filter out sentences that match the proposition pattern
      filtered_sentences = [sentence for sentence in sentences if not re.search(proposition_pattern, sentence, re.IGNORECASE)]
      
      # Join the remaining sentences back into a single string
      return ' '.join(filtered_sentences)
  
  # Sample text for testing
  sample_text = "Python is a great programming language. It is widely used in data science. Do you like coding? She believes in the power of technology. The sky appears blue."
  cleaned_text = remove_propositions(sample_text)
  print(cleaned_text)

   

Output:

Do you like coding?