owusu/DS-LLM-TEMPLATE-FINETUNING
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d49b4ff2d5b0c7d6a4e7926c18863f19ea13bbe3
DS-LLM-TEMPLATE-FINETUNING/unsloth_compiled_cache/__pycache__/UnslothXPOTrainer.cpython-310.pyc
T

228 lines
29 KiB
Plaintext
Raw Normal View History

instruct model setup
2025-08-28 17:57:59 +00:00
o
?—°heÃã@sºdZddlmZddlZddlmZddlmZddlmZm Z m
Z
m Z m Z m
Z
mZmZddlmZmZmZmZmZmZmZmZmZmZmZm
Z
mZmZmZmZmZmZm Z mZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)mZm*Z*m+Z+m,Z,mZm-Z-m.Z.ddl*Z*ddlTddl/m0Z0m1Z1dd l2m3Z3ddlZddl4Z5dd
l6m7Z7ddlmZdd l8m9Z9m:Z;d d
d d
d
dœZ<ej=d d e<d�dd„ƒZ>e0Gdd„deƒƒZ? Gdd„deƒZ@Gdd„de@ƒZAdS)z9
2025.8.9
2025.8.10
4.55.4
0.21.0
__UNSLOTH_VERSIONING__
é)ÚTensorN)Ú
functional)ÚAnyÚListÚOptionalÚTupleÚUnionÚDictÚSetÚCallable)%rÚBaseImageProcessorÚBasePairwiseJudger ÚDatasetÚEvalPredictionÚFÚFeatureExtractionMixinÚIterableDatasetÚOnlineDPOTrainerÚOptimizerNamesrÚ PeftModelÚPreTrainedModelÚPreTrainedTokenizerBaseÚProcessorMixinÚSIMPLE_CHAT_TEMPLATEÚTrainerCallbackrÚ XPOConfigÚ
XPOTrainerÚ empty_cacheÚgenerate_model_cardÚget_comet_experiment_urlÚ
get_rewardÚis_conversationalÚis_peft_availableÚis_wandb_availableÚjinja2Úmaybe_apply_chat_templateÚnnÚosÚselective_log_softmaxÚtextwrapÚtorchÚtruncate_rightÚunwrap_model_for_generation)Ú*)Ú dataclassÚfield)ÚVersion)Ú nullcontext)ÚDataCollatorForSeq2SeqÚDataCollatorForLanguageModelingTF)Úepilogue_fusionÚ max_autotuneÚ
shape_paddingz
trace.enabledztriton.cudagraphs)ÚdynamicÚ fullgraphÚoptionsc
Cs¾tj| d|jd¡ddd�}tj| d¡ddd�}g}t||ƒD](\}}| tj¡}tj|d| d¡d�  d¡}tj
|dd�}||} |  | ¡q! t  |¡}| |jd|jdf¡}|S)Néÿÿÿÿér)ÚchunksÚdim)r=Úindex©r=é)
r*ÚchunkÚreshapeÚshapeÚzipÚtoÚfloat32ÚgatherÚ unsqueezeÚsqueezeÚ logsumexpÚappendÚconcat)
Úlogitsr>Úchunked_logitsÚ
chunked_indexÚall_per_token_logpsÚ chunk_logitsÚ chunk_indexÚselected_logitsÚlogsumexp_valuesÚper_token_logps©rVúQ/workspace/DS-LLM-TEMPLATE-FINETUNING/unsloth_compiled_cache/UnslothXPOTrainer.pyÚchunked_selective_log_softmax"s  
rXcsšeZdZUdZedddid�Zeeed<edddid�Z ee
ed <eddd
id�Z ee
ed <  
                            ! ! " #     $           $      % &  '         (      #    $   ) *         + ,   -   . /     d2‡fd0d1„ Z ‡Z
S)3ÚUnslothXPOConfiga­
Configuration class for the [`XPOTrainer`].
Subclass of [`OnlineDPOConfig`] we can use all its arguments and add the following:
Parameters:
alpha (`float` or `list[float]`, *optional*, defaults to `1e-5`):
Weight of the XPO loss term. If a list of floats is provided then the alpha is selected for each new epoch
and the last alpha is used for the rest of the epochs.
NÚhelpzvLLM SamplingParams)ÚdefaultÚmetadataÚvllm_sampling_paramsr:z8Chunk size to reduce memory usage. -1 is most efficient.Úunsloth_num_chunksz'Maximum sequence length to truncate to.Úmax_seq_lengthFÚnor;éréúç-Cëâ6
?ç{®Gáz„?çÍÌÌÌÌÌì?ç+‡ÙÎ÷ï?ç:Œ0âŽyE>çð?ç@Úlinearçš™™™™™¹?ÚpassiveÚwarningTÚstepsr@éôéO
ÚO1ÚautoÚçÚ
adamw_8bitÚlengthÚ
every_saveÚlastéé@éÚsigmoidÚvllmçš™™™™™á?c’” s|dkr td|›d�ƒ‚|dkrtd|›d�ƒ‚|dur(|#dkr(|$dkr(d}d }#|ˆdur:d
d lm}“t|“ƒd d
ƒ}ˆ|…d
krBtdƒ‚|…dkrJtdƒ‚tƒjdŸid|“d|“d|“d|“d|“d|“d|“d|“d| “d|
“d| “d| “d|
“d|“d|“d |“d!|“d"|“d#|“d$|“d%|“d&|“d'|“d(|“d)|“d*|“d+|“d,|“d-|“d.|“d/|“d0| “d1|!“d2|"“d3|#“d4|$“d5|%“d6|&“d7|'“d8|(“d9|)“d:|*“d;|+“d<|,“d=|-“d>|.“d?|/“d@|0“dA|1“dB|2“dC|3“dD|4“dE|5“dF|6“dG|7“dH|8“dI|9“dJ|:“dK|;“dL|<“dM|=“dN|>“dO|?“dP|@“dQ|A“dR|B“dS|C“dT|D“dU|E“dV|F“dW|G“dX|H“dY|I“dZ|J“d[|K“d\|L“d]|M“d^|N“d_|O“d`|P“da|Q“db|R“dc|S“dd|T“de|U“df|V“dg|W“dh|X“di|Y“dj|Z“dk|[“dl|\“dm|]“dn|^“do|_“dp|`“dq|a“dr|b“ds|c“dt|d“du|e“dv|f“dw|g“dx|h“dy|i“dz|j“d{|k“d||l“d}|m“d~|n“d|o“d€|p“d�|q“d‚|r“dƒ|s“d„|t“d…|u“d†|v“d‡|w“dˆ|x“d‰|y“dŠ|z“d‹|{“dŒ||“d�|}“dŽ|~“d�|“d�|€“d‘|�“d’|‚“d“|ƒ“d”|„“d•|…“d–|†“d—|‡“d˜|ˆ“d™|‰“dš|Š“d›|‹“dœ|Œ“d�|�“dž|Ž“|’¤Ž|�|_|�|_ |‘|_
dS) NgH¯¼šò×z>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!r@za` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!rnroÚunsloth_training_checkpointsr`r)Ú cpu_countr;razUUnsloth: Please set a positive non-zero temperature since your results will be wrong.é
zgUnsloth: Please set a positive non-zero temperature less than 10, since sampling will be quite erratic.Ú
output_dirÚoverwrite_output_dirÚdo_trainÚdo_evalÚ
do_predictÚ
eval_strategyÚprediction_loss_onlyÚper_device_train_batch_sizeÚper_device_eval_batch_sizeÚper_gpu_train_batch_sizeÚper_gpu_eval_batch_sizeÚgradient_accumulation_stepsÚeval_accumulation_stepsÚ
eval_delayÚtorch_empty_cache_stepsÚ
learning_rateÚ weight_decayÚ
adam_beta1Ú
adam_beta2Ú adam_epsilonÚ
max_grad_normÚnum_train_epochsÚ max_stepsÚlr_scheduler_typeÚ warmup_ratioÚ warmup_stepsÚ log_levelÚlog_level_replicaÚlog_on_each_nodeÚ logging_dirÚlogging_strategyÚlogging_first_stepÚ
logging_stepsÚlogging_nan_inf_filterÚ
save_strategyÚ
save_stepsÚsave_total_limitÚsave_safetensorsÚsave_on_each_nodeÚsave_only_modelÚ'restore_callback_states_from_checkpointÚno_cudaÚuse_cpuÚuse_mps_deviceÚseedÚ data_seedÚ
jit_mode_evalÚuse_ipexÚbf16Úfp16Úfp16_opt_levelÚhalf_precision_backendÚbf16_full_evalÚfp16_full_evalÚtf32Ú
local_rankÚ ddp_backendÚ
tpu_num_coresÚtpu_metrics_debugÚdebugÚdataloader_drop_lastÚ
eval_stepsÚdataloader_num_workersÚdataloader_prefetch_factorÚ
past_indexÚrun_nameÚ disable_tqdmÚremove_unused_columnsÚ label_namesÚload_best_model_at_endÚmetric_for_best_modelÚgreater_is_betterÚignore_data_skipÚfsdpÚfsdp_min_num_paramsÚ fsdp_configÚ"fsdp_transformer_layer_cls_to_wrapÚaccelerator_configÚ deepspeedÚlabel_smoothing_factorÚoptimÚ
optim_argsÚ adafactorÚgroup_by_lengthÚlength_column_nameÚ report_toÚddp_find_unused_parametersÚddp_bucket_cap_mbÚddp_broadcast_buffersÚdataloader_pin_memoryÚdataloader_persistent_workersÚskip_memory_metricsÚuse_legacy_prediction_loopÚ push_to_hubÚresume_from_checkpointÚ hub_model_idÚ hub_strategyÚ hub_tokenÚhub_private_repoÚhub_always_pushÚ hub_revisionÚgradient_checkpointingÚgradient_checkpointing_kwargsÚinclude_inputs_for_metricsÚeval_do_concat_batchesÚ fp16_backendÚpush_to_hub_model_idÚpush_to_hub_organizationÚpush_to_hub_tokenÚ
mp_parametersÚauto_find_batch_sizeÚfull_determinismÚ torchdynamoÚ ray_scopeÚ ddp_timeoutÚ
torch_compileÚtorch_compile_backendÚtorch_compile_modeÚinclude_tokens_per_secondÚinclude_num_input_tokens_seenÚneftune_noise_alphaÚoptim_target_modulesÚbatch_eval_metricsÚ
eval_on_startÚuse_liger_kernelÚliger_kernel_configÚeval_use_gather_objectÚaverage_tokens_across_devicesÚreward_model_pathÚjudgeÚmax_new_tokensÚ
max_lengthÚ temperatureÚmissing_eos_penaltyÚ loss_typeÚdataset_num_procÚdisable_dropoutÚuse_vllmÚvllm_model_implÚgpu_memory_utilizationÚds3_gather_for_generationÚmodel_init_kwargsrV) ÚFloatingPointErrorÚ
OverflowErrorÚmultiprocessingr€ÚmaxÚ MathErrorÚsuperÚ__init__r]r^r_)”Úselfr‚rƒr„r…r†r‡rˆr‰rŠr‹rŒr�rŽr�r�r‘r’r“r”r•r–r—r˜r™ršr›rœr�ržrŸr r¡r¢r£r¤r¥r¦r§r¨r©rªr«r¬r­r®r¯r°r±r²r³r´rµr¶r·r¸r¹rºr»r¼r½r¾r¿rÀrÁrÂrÃrÄrÅrÆrÇrÈrÉrÊrËrÌrÍrÎrÏrÐrÑrÒrÓrÔrÕrÖr×rØrÙrÚrÛrÜrÝrÞrßràrárârãrärårærçrèrérêrërìrírîrïrðrñròrórôrõrör÷rørùrúrûrürýrþrÿrrrrrrrrrr r
r r r
rrr]r^r_Úkwargsr€©Ú __class__rVrWrMs˜  ÿþýüûúùø ÷
ö õ ô
óòñðïîíìëêéèçæåäãâá à!ß"Þ#Ý$Ü%Û&Ú'Ù(Ø)×*Ö+Õ,Ô-Ó.Ò/Ñ0Ð1Ï2Î3Í4Ì5Ë6Ê7É8È9Ç:Æ;Å<Ä=Ã>Â?Á@ÀA¿B¾C½D¼E»FºG¹H¸I·JKµL´M³N²O±P°Q¯R®S­T¬U«VªW©X¨Y§Z¦[¥\¤]£^¢_¡` aŸbžc�dœe›fšg™h˜i—j–k•l”m“n’o‘p�q�rŽs�tŒu‹vŠw‰xˆy‡z†{…|„}ƒ~‚��ÿ�þ�ý�ü�û�ú�ù�ø �÷
�ö �õ �ô
�ó�ò�ñ
zUnslothXPOConfig.__init__)‘NNFFFr`Fr;r;NNrararrbrcrdrerfrgrhrir:rjrkrrlrmTNrnFr@FrnroNTFFFFFFrprpFFFFrqrrFFNr:NNFrsFNrNr:NNTNFNNFrsrNNNNrtruNFFrvNNNNTFTFFNNrwNNFNFNFTrrNNNrsTFNrxryFNNFFNNFFFNFTNNrzr{reNr|NTFr}r~TNNr:N)Ú__name__Ú
__module__Ú __qualname__Ú__doc__r/r]rrÚ__annotations__r^Úintr_rÚ
__classcell__rVrVrrWrY3sB
 þþþ�írYc sÒeZdZdZddgZ              d0deeejfdeeejfde ejd e e
d
e e d e e d e ee
efd
e ee
eee
ffde eeeeefde ede e egefde eedeejjejjjfde e ejejgejfddf‡fdd„
Ze dd„ƒZ!dd„Z"dd„Z#dd„Z$dd „Z%d!d"„Z&d#d$„Z'  dd%d&„Z( d1dejd'eeeeje)ffd(e e*dejfd)d*„Z+   d2d+e ed,e ed-eeeedffd.d/„Z,‡Z-S)3Ú_UnslothXPOTrainerrsÚtrlÚxpoN©NNÚmodelÚ ref_modelÚ reward_modelrÚargsÚ
data_collatorÚ
train_datasetÚ eval_datasetÚprocessing_classÚ peft_configÚcompute_metricsÚ callbacksÚ
optimizersÚpreprocess_logits_for_metricsÚreturncsˆtƒj||||||||| | |
| | |
|d�|jj|_ggggggggggggggdœ|_|jdurBg|jd<g|jd<g|jd<dSdS)N)r&r'rr(r)r*r+r,r-Úreward_processing_classr.r/r0r1r2)úloss/dpoúloss/xpoú objective/klúobjective/entropyúrewards/chosenúrewards/rejectedúrewards/accuraciesúrewards/marginsú logps/chosenúlogps/rejectedúval/model_contain_eos_tokenúval/ref_contain_eos_tokenÚalphaÚbetaúobjective/model_scoresúobjective/ref_scoresúobjective/scores_margin)rrr)rAÚ_alphaÚstatsr()rr&r'r(rr)r*r+r,r-r.r/r0r1r2rrVrWr‰sLñ
ï


üz_UnslothXPOTrainer.__init__cCs<t|jtƒr|jj}|t|jƒkr|j|S|jdS|jS)Nr:)Ú
isinstancerFÚlistÚstateÚepochÚlen)rrKrVrVrWrAÊs "z_UnslothXPOTrainer.alphac CsÞt||jƒ�}|j|d|d|jd�}Wdƒn1swY|jdur>|j |¡}tƒr;t|tƒr;|  ¡}n
|}n|j |j¡}t||jƒ�}|j|d|d|jd�}Wdƒ||fS1sfwY||fS)NÚ input_idsÚattention_mask)rMrNÚgeneration_config)
r,Ú acceleratorÚgeneraterOr'Ú unwrap_modelr"rHrÚget_base_model) rÚpromptsr&Úunwrapped_policy_model_for_genÚ model_outputÚ"unwrapped_main_model_for_ref_logicÚactual_model_for_ref_generationÚfinal_ref_model_for_genÚ
ref_outputrVrVrWÚ_generate_completionsÒs.ýÿ
 
ý
ÿùz(_UnslothXPOTrainer._generate_completionsc CsÖ|djd}|dd…|d…f}t||jj|jjƒ\}}tj|d|fdd�tj|d|fdd�|ddœ}|dd…|d…f}t||jj|jjƒ\}} tj|d|fdd�tj|d| fdd�|ddœ}
||
fS)NrMr@r?rNÚraw©rMrNr\)rCr+r-Ú eos_token_idÚ pad_token_idr*Úcat) rrVrZrTÚcontext_lengthÚmodel_completion_idsÚmodel_completion_maskÚ
model_dataÚref_completion_idsÚref_completion_maskÚref_datarVrVrWÚ_process_completionsîs$ÿýÿýz'_UnslothXPOTrainer._process_completionsc CsØt ¡�&t|j|d|jj|ƒ\}}}t|j|d|jj|ƒ\}}}Wdƒn1s-wY|jjdurhtj|d|jj kdd�}tj|d|jj kdd�}|||jj8<|||jj8<||fS)NrMr:r?)
r*Úno_gradr r(r-r_r)rÚanyr^) rrdrgraÚ_Ú model_scoresÚ
ref_scoresÚmodel_contain_eosÚref_contain_eosrVrVrWÚ_compute_rewards s

ÿ ÿü z#_UnslothXPOTrainer._compute_rewardsc s|d}|jj|ddd…|d…fdd�}dd„|Dƒ}|jj|ddd…|d…fdd�}dd„|Dƒ}td|d iƒrpd
d„|Dƒ}t ¡}| t¡‰‡fd d„|Dƒ}‡fd d„|Dƒ}d
d„|Dƒ}‡fdd„|Dƒ}|j |tt ||ƒƒ¡}t
j dd„|Dƒ|dj d�S)Nr\rMT)Úskip_special_tokenscSóg|]}| ¡‘qSrV©Ústrip©Ú.0Ú
completionrVrVrWÚ
<listcomp> óz5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>cSrrrVrsrurVrVrWrx%ryÚpromptrcSóg|]}d|dœg‘qS©Ú assistant)ÚroleÚcontentrVrurVrVrWrx(ó ÿcóg|]}ˆj|d�‘qS©)Úmessages©Úrender)rvÚmessage©ÚtemplaterVrWrx-ócr�r‚r„rur‡rVrWrx.r‰cSr{r|rVrurVrVrWrx0r€cr�r‚r„rur‡rVrWrx3r‰cSsg|]}|dk‘qS)rrV)rvÚrankrVrVrWrx<ry)Údevice)
r-Ú batch_decoder!r$Ú EnvironmentÚ from_stringrrrIrDr*Útensorr‹) rrdrgrarTÚmodel_data_completionsÚref_data_completionsÚ environmentÚranks_of_first_completionrVr‡rWÚ_compute_judges4ÿÿÿ
ÿ þz!_UnslothXPOTrainer._compute_judgec s‡fdd„}|||ƒ}|||ƒ}t ¡�8|jdur9| ¡�|||ƒ}|||ƒ} Wdƒn1s3wYn ||j|ƒ}||j|ƒ} Wdƒn1sOwY|ddd…ˆd…fdk}
|ddd…ˆd…fdk} | |
d¡}| | d¡}|  | d¡} | |
d¡}||| |fS)NcsP||d|dd�}|jdd…ˆdd…f}t||ddd…ˆd…fƒ}|S)NrMrN)rNr@r:)rMr()ÚmÚdataÚoutputrMÚtoken_logprobs©rarVrWÚcompute_logprobs_for_data?szG_UnslothXPOTrainer._compute_logprobs.<locals>.compute_logprobs_for_datarNrrt)r*rir'Údisable_adapterÚ masked_fill) rr&rdrgraršÚmodel_logprobs_model_dataÚmodel_logprobs_ref_dataÚref_logprobs_model_dataÚref_logprobs_ref_dataÚmodel_padding_maskÚref_padding_maskrVr™rWÚ_compute_logprobs>s* 





 þ€  €ù
     z$_UnslothXPOTrainer._compute_logprobscCsò| d¡}| d¡}| d¡}| d¡} t |||¡}
t || |¡} |
| } t |||¡}
t || |¡}|
|}| |}|jjdkrNt |j|¡ }n|jjdkr`|dd|jd}n td|jj›�ƒ‚|j |}|| 
¡}|||fS)Nr@r|Úiporazinvalid loss type ) Úsumr*Úwherer)rrÚ
logsigmoidrBÚNotImplementedErrorrAÚmean)rr�ržr rŸÚ chosen_maskÚmodel_logprobs_model_data_sumÚmodel_logprobs_ref_data_sumÚref_logprobs_ref_data_sumÚref_logprobs_model_data_sumÚchosen_model_logprobsÚchosen_ref_logprobsÚchosen_log_ratiosÚrejected_model_logprobsÚrejected_ref_logprobsÚrejected_log_ratiosrMÚ
dpo_lossesÚ
xpo_lossesÚlossrVrVrWÚ_compute_losses^s&


  
 
z"_UnslothXPOTrainer._compute_lossesc
$sâ‡fdd„}
ˆjd |
|ƒ¡ˆjd |
| ƒ¡ˆjdur?ˆjd |
| ƒ¡ˆjd |
| ƒ¡ˆjd |
| | ƒ¡| d¡}| d¡}| d¡}| d¡}t |||¡}t |||¡}||}t |||¡}t |||¡}||}ˆjd  |
| ¡| ¡ƒ¡ˆjd
 |
| ¡| ¡ƒ¡|ˆj}|ˆj}ˆjd  |
| ¡ƒ¡ˆjd  |
| ¡ƒ¡||}||}| d¡| d¡ ¡d
}ˆjd |
|ƒ¡| d¡ }| d¡ }| ¡| ¡d
}ˆjd |
|ƒ¡||} ˆjd |
|  ¡ƒ¡| dk ¡}!ˆjd |
|! ¡ƒ¡|ddd…|
d…fˆj j
kj dd�}"|ddd…|
d…fˆj j
kj dd�}#ˆjd |
|" ¡ƒ¡ˆjd |
|# ¡ƒ¡ˆjd ˆj ¡ˆjd ˆj¡dS)Ncsˆj |¡ ¡ ¡S©N)rPÚgather_for_metricsr©Úitem)r�©rrVrWÚ gather_mean–sz7_UnslothXPOTrainer._log_statistics.<locals>.gather_meanr5r6rCrDrEr@r=r>r9r:rar7r8r<rr;rMr?r?r@rArB)
rGrKr(r¥r*r¦r©rBÚfloatr-r^rjrA)$rrdrgr�ržr rŸrªrµr¶rarlrmr½r«r¬r­r®r¯r°r±r²r³r´Úchosen_rewardsÚrejected_rewardsÚ
kl_model_dataÚ kl_ref_dataÚmean_klÚentropy_model_dataÚentropy_ref_dataÚ mean_entropyÚmarginÚaccuracyÚ model_eosÚref_eosrVr¼rWÚ_log_statistics†sR 




  

   ((z"_UnslothXPOTrainer._log_statisticsÚinputsÚnum_items_in_batchcs| ¡tttˆ ¡ƒƒƒ}ˆd}‡fdd„t|ƒDƒ‰‡fdd„ˆDƒ‰‡fdd„ˆDƒ‰ˆ ˆ¡‰ˆ ˆ¡‰ˆdjd}ˆdˆd|d œ}Šˆ  ||¡\}}ˆ 
|||¡\} }
ˆj duroˆ  | |
|¡\} } | | k}
n d
\} } ˆ 
| |
|¡}
ˆ || |
|¡\}}}}ˆ |||||
¡\}}}ˆ | |
| ¡| ¡|||
| ¡| ¡|| | ¡ ˆjjdur½ˆjjˆjjd kr½tƒi}ˆjjtjtjfvrψ ¡|d <ˆjjdkrÙ| ¡}ˆjrøt  |ˆj!¡� }| "¡Wdƒn1sòwYn
ˆj#j"|fi|¤Ž| ¡ˆjj$S)
Nrzcs"g|]
‰‡fdd„ˆ ¡Dƒ‘qS)csi|] \}}||ˆ“qSrVrV)rvÚkÚv©ÚirVrWÚ
<dictcomp>ász?_UnslothXPOTrainer.training_step.<locals>.<listcomp>.<dictcomp>)Úitems)rv)rÌrÐrWrxáó"z4_UnslothXPOTrainer.training_step.<locals>.<listcomp>csg|]}t|ˆjƒ‘qSrV)r%r-©rvÚxr¼rVrWrxâr‰cs"g|]
}ˆ |ˆjjjˆj¡‘qSrV)Ú tokenize_rowr&ÚconfigÚis_encoder_decoderr-rÕr¼rVrWrxãrÔÚprompt_input_idsr@Úprompt_attention_maskr]r%rr‘)%ÚtrainrLÚnextÚiterÚvaluesÚranger*Ú_prepare_inputsrCr[rhr(rpr”r£r¸rËÚdetachr)r�rJÚ global_steprrÒrÚLOMOÚADALOMOÚ_get_learning_rateÚn_gpur©Úuse_apexÚampÚ
scale_lossÚ optimizerÚbackwardrPr�)rr&rÌrÍÚ
batch_sizerTrarVrZrdrgrlrmrªr�ržr rŸr·rµr¶rÚ scaled_lossrV)rÌrrWÚ
training_stepÙsv

ý


ÿ
û ô   
ÿ€z _UnslothXPOTrainer.training_stepÚ
model_nameÚ dataset_nameÚtagsc
Csê| ¡sdSt|jjdƒrtj |jjj¡s|jjj}nd}|dur&tƒ}n
t |t
ƒr/|h}nt|ƒ}t|jjdƒr?|  d¡|  |j
¡t d¡}t|||j||tƒr]tjdur]tjjndtƒd|ddd � }| tj |jjd
¡¡dS) aî
Creates a draft of a model card using the information available to the `Trainer`.
Args:
model_name (`str` or `None`, *optional*, defaults to `None`):
Name of the model.
dataset_name (`str` or `None`, *optional*, defaults to `None`):
Name of the dataset used for training.
tags (`str`, `list[str]` or `None`, *optional*, defaults to `None`):
Tags to be associated with the model card.
NÚ
_name_or_pathÚunsloth_versionÚunslotha� @article{jung2024binary,
title = {{Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF}},
author = {Tengyang Xie and Dylan J. Foster and Akshay Krishnamurthy and Corby Rosset and Ahmed Awadallah and Alexander Rakhlin},
Reference in New Issue Copy Permalink