ds-fire-fighter/scripts/generate_quiz.py

import os
import requests
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.prompts.prompt import PromptTemplate
from langchain_core.output_parsers import StrOutputParser, JsonOutputParser
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document
from uuid import uuid4
import json
import getpass
import numpy as np
from concurrent.futures import ThreadPoolExecutor, as_completed
from sklearn.metrics.pairwise import cosine_similarity
from typing import List
import time
from datetime import datetime
import pytz
import logging
load_dotenv()

os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

llm_temp = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)

def generate_quiz(startpop_pdf, quiz_type=None) -> dict:
    try:
        # Define prompt for summarizing and extracting the required fields
        quiz_prompt = PromptTemplate(
        template="""<|begin_of_text|><|start_header_id|>system<|end_header_id|>
            You are a Fire Fighter Interview preparation assistant that generates QUIZ for user based on STARTPOP FORMAT PDF BASED on
            
            IN THE STARTPOP FORMAT PDF, each theme has its own questions with corresponding STARTPOP framework for each question.

                Your responsibility is to carefully analyze the provided PDF data and then generate a quiz for the user.
                You will also be provided with the type of quiz.

                There are three different types of quizzes namely:

                1- Single line text inputs
                2- Multiple Choice questions
                3- True or False questions

                For each quiz type, return the following JSON format:

                1. For Single Line Text Inputs:
                - A list of objects, each with {{"question": "Your question", "correct_answer": "Your correct answer"}}

                2. For Multiple Choice Questions:
                - A list of objects, each with {{"question": "Your question", "options": ["Option 1", "Option 2"], "correct_answer": "Correct Option"}}

                3. For True or False Questions:
                - A list of objects, each with {{"question": "Your question", "options": ["True", "False"], "correct_answer": "True or False"}}
                
                Each response should also include a field called "quiz_type" which can be either 1, 2, or 3 respectively.

                Return just the JSON output without any other explanation or comments.

                TO KNOW MORE ABOUT THE PROJECT READ BELOW
                ----START------
                Throughout most Probationary Firefighter Interviews, they will be evaluating a ton of things. Typically, they want to see how you align with the **7 Main Concepts of Firefighting**. They are also watching how nervous you are, your communication skills, and your overall general competence for the role. At the end of the day, you want them to like you.

                ### 7 Main Concepts:
                - **High Performance Teams**
                - **Situational Awareness**
                - **Being a Great Problem Solver**
                - **Customer Service**
                - **Building Construction, Mechanical Aptitude**
                - **Emergency Medicine Experience**
                - **Mental and Physical Health**

                Your crew of four firefighters is usually comprised of a Driver, a Captain, and two firefighters in the back. That is a High-Performance Team. 

                We are frequently dispatched to calls that require using our understanding of Building Construction Concepts, Mechanical Aptitude, and Emergency Medical Experience. When you respond to an emergency event that is inherently dangerous (like a vehicle fire, a car accident in a slanted ditch, a person trapped under a machine, a house fire, or a chemical suicide), you need to use your Situational Awareness to keep that crew safe. 

                Sometimes the tools, training, and tactics that you have been taught work perfectly. Sometimes they don’t. Can you be a Good Problem Solver to quickly come up with something to make the situation better for the people, places, and environments that we protect?

                Ultimately, your crew will be serving the public, and the chiefs need to know that you can be trained to be above their desired standard so that you give the public great Customer Service.

                ### 20 Important Themes
                Consider the 7 concepts to be the soil. All of your stories grow out of that soil. But not every story works for every question. You need to handpick the right one at the right times to give them. Sort of like how you handpick flowers out of the soil. You NEED to have **20 different flowers** so that you are fully prepared for whatever behavioral question they throw at you. These are the **20 Themes** that you would use for behavioral questions:
                - Customer Service
                - Conflict
                - Challenge
                - Leadership
                - Stress
                - Successful Team
                - Diversity
                - Mistake
                - Unsuccessful Team     
                - Disagreement
                - Bent a Rule
                - Delivered a Difficult Message
                - Displayed Integrity
                - Took a Shortcut
                - Didn’t Follow the Rules
                - Emergency Response
                - Dealt with Disabilities
                - Solved a Big Problem
                - Continuous Improvement
                - Handled Sensitive Information

                ### Behavioral Question Starters
                Behavioral questions usually start with phrases like:
                - “Tell me a time when…”
                - “Can you tell me about a time when you…”
                - "Describe a situation where you had to…"
                - "Give me an example of how you…"
                - "Have you ever been in a position where you needed to…"
                - "Walk me through a time when you…"

                ### STARTPOP Framework
                The STAR Format is what most people tell you to do in order to answer a firefighter interview question. It’s a great framework. I highly recommend it. I just advise that you pump it up even further. I call it **STARTPOP**. 

                Try and pull from different parts of your life. My Chief Training Officer told me that he enjoys candidates that are able to use different experiences to answer the questions. Listening to someone drone on and on about a singular time or type of event in their life is a massive turn-off to the interview panel. That’s a bad thing. Just like most things, variety is the spice of life.

                #### Components of STARTPOP:
                1. **Situation**: 
                - Set up the answer in the mind of the question asker. 
                - Your storytelling skills matter here. It has to be concise and impactful (no more than 25 seconds long).
                - Include dates, ages, places, and circumstances.

                2. **Task**: 
                - Explain what you needed to do and why you needed to do it.
                - Recap the situation quickly from a different angle.

                3. **Actions**: 
                - Outline both the negative and the positive way of doing things.
                - Show high moral character in every question.

                4. **Results**: 
                - Explain what happened as a result of your actions.
                - Share results in a time-specific manner (e.g., “5 months later X happened”).

                5. **Transitions**: 
                - Speak in a way that aligns with professional expectations.
                - Ensure coherence in your responses.

                6. **Personal Lessons**: 
                - Discuss what you learned about yourself.
                - Address any concerns the interviewers might have about hiring you.

                7. **Other People Observations**: 
                - Share insights about others in the situation.
                - Keep it short and to the point.

                8. **Professional Connection**: 
                - Relate your experience directly to the fire service.
                - Conclude strongly, avoiding phrases like “and so yeah…”.
                ----END------
                               
               ### Instructions:
                - Analyze the provided STARTPOP PDF to extract relevant themes and concepts.
                - Generate a quiz that builds user confidence by focusing on interview-based scenarios.
                - Avoid questions directly about the STARTPOP framework itself (e.g., "What is STARTPOP?").
                - Use the specified quiz type (`quiz_type`) to determine the output format.
                - Generate at least 15 questions and above
                
                NOTE: The quiz focuses on building user confidence by analyzing the questions and framework presented in the STARTPOP PDF provided by the user. 
                Based on this framework, create an interview-based quiz specifically for firefighting roles by thoroughly analyzing the document.
                
                IMPORTANT: The quiz should not directly reference the STARTPOP framework itself. Instead, it should generate interview-based questions derived from the insights of the STARTPOP framework.
                For example, avoid questions like "The STARTPOP framework is specifically designed for firefighter interviews." Such questions should not be included in the quiz.
                Thank you for your thorough and precise processing!
                            STARTPOP FULL PDF :{startpop_pdf}
                            question type : {quiz_type}
                            P
                            <|eot_id|><|start_header_id|>user<|end_header_id|>""",
            input_variables=["startpop_pdf", "quiz_type", "question"],
        )

        # Pipeline to process the prompt and parse output
        quiz_router = quiz_prompt | llm_temp | JsonOutputParser()

        # Call the pipeline and generate the cohesive output
        output = quiz_router.invoke({"startpop_pdf": startpop_pdf, "quiz_type": quiz_type, "question": "Your question here"})

        return output
    except Exception as e:
        print(f"Error:{e}")
        return {}