ds-fire-fighter/scripts/generate_quiz.py

import os
import requests
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from langchain_core.prompts.prompt import PromptTemplate
from langchain_core.output_parsers import StrOutputParser, JsonOutputParser
from langchain_openai import OpenAIEmbeddings
from langchain_core.documents import Document
from uuid import uuid4
import json
import getpass
import numpy as np
from concurrent.futures import ThreadPoolExecutor, as_completed
from sklearn.metrics.pairwise import cosine_similarity
from typing import List
import time
from datetime import datetime
import pytz
import logging
load_dotenv()

os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

llm_temp = ChatOpenAI(model="gpt-4o-mini", temperature=0.7,max_tokens=10000)
def generate_quiz(startpop_pdf, quiz_type=None) -> dict:
    try:
        # Define the prompt template for generating the quiz
        quiz_prompt = PromptTemplate(
            template="""
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an assistant designed to generate firefighter interview quizzes based on the STARTPOP framework provided in a PDF document. Your task is to analyze the content of the PDF and create a quiz tailored to the specified quiz type. 

### Quiz Types:
1. **Single Line Text Inputs**: 
   - Output format: `{"question": "Your question", "correct_answer": "Your correct answer"}`

2. **Multiple Choice Questions**:
   - Output format: `{"question": "Your question", "options": ["Option 1", "Option 2"], "correct_answer": "Correct Option"}`

3. **True or False Questions**:
   - Output format: `{"question": "Your question", "options": ["True", "False"], "correct_answer": "True or False"}`

Each quiz must include a field called `"quiz_type"` with values `1`, `2`, or `3` corresponding to the quiz type.

### Project Overview:
Firefighter interviews evaluate candidates based on **7 Main Concepts** and **20 Important Themes**. These are used to assess alignment with firefighting principles, communication skills, problem-solving abilities, and overall competence. 

#### 7 Main Concepts:
- High Performance Teams
- Situational Awareness
- Being a Great Problem Solver
- Customer Service
- Building Construction & Mechanical Aptitude
- Emergency Medicine Experience
- Mental & Physical Health

#### 20 Important Themes:
- Customer Service
- Conflict
- Challenge
- Leadership
- Stress
- Successful Team
- Diversity
- Mistake
- Unsuccessful Team
- Disagreement
- Bent a Rule
- Delivered a Difficult Message
- Displayed Integrity
- Took a Shortcut
- Didn’t Follow the Rules
- Emergency Response
- Dealt with Disabilities
- Solved a Big Problem
- Continuous Improvement
- Handled Sensitive Information

#### Behavioral Question Starters:
Questions often begin with phrases like:
- "Tell me a time when..."
- "Can you tell me about a time when you..."
- "Describe a situation where you had to..."
- "Give me an example of how you..."
- "Have you ever been in a position where you needed to..."
- "Walk me through a time when you..."

#### STARTPOP Framework:
The STARTPOP framework enhances the STAR method by adding depth and variety to responses. Its components are:
1. **Situation**: Set up the context (dates, places, circumstances).
2. **Task**: Explain what needed to be done and why.
3. **Actions**: Outline both positive and negative approaches.
4. **Results**: Share outcomes in a time-specific manner.
5. **Transitions**: Ensure professional coherence.
6. **Personal Lessons**: Discuss what you learned.
7. **Other People Observations**: Share insights about others involved.
8. **Professional Connection**: Relate the experience to firefighting.

### Instructions:
- Analyze the provided STARTPOP PDF to extract relevant themes and concepts.
- Generate a quiz that builds user confidence by focusing on interview-based scenarios.
- Avoid questions directly about the STARTPOP framework itself (e.g., "What is STARTPOP?").
- Use the specified quiz type (`quiz_type`) to determine the output format.

STARTPOP FULL PDF: {startpop_pdf}
QUIZ TYPE: {quiz_type}

<|eot_id|><|start_header_id|>user<|end_header_id|>
""",
            input_variables=["startpop_pdf", "quiz_type"],
        )

        # Pipeline to process the prompt and parse the output
        quiz_router = quiz_prompt | llm_temp | JsonOutputParser()

        # Call the pipeline and generate the cohesive output
        output = quiz_router.invoke({"startpop_pdf": startpop_pdf, "quiz_type": quiz_type})
        return output

    except Exception as e:
        print(f"Error: {e}")
        return {}