242 lines
51 KiB
Plaintext
242 lines
51 KiB
Plaintext
|
§
|
|||
|
|
5$�hÀ ã ó — d Z ddlmZ ddlZddlmZ ddlmZ ddlmZm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z m
|
|||
|
|
Z
|
|||
|
|
mZmZ ddlmZmZmZmZmZmZmZmZmZmZmZm
|
|||
|
|
Z
|
|||
|
|
mZmZmZm Z m Z m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)mZm*Z*m+Z+m,Z,mZm-Z-m.Z. ddl*Z*ddlT ddl/m0Z0m1Z1 dd l2m3Z3 ddlZddl4Z5dd
|
|||
|
|
l6m7Z7 ddlmZ dd l8m9Z9m:Z; d d
|
|||
|
|
d d
|
|||
|
|
d
|
|||
|
|
dœZ< ej= d d e<¬¦ « d„ ¦ « Z>e0 G d„ de ¦ « ¦ « Z? G d„ de¦ « Z@ G d„ de@¦ « ZAdS )z8
|
|||
|
|
2025.8.4
|
|||
|
|
2025.8.5
|
|||
|
|
4.55.1
|
|||
|
|
0.21.0
|
|||
|
|
__UNSLOTH_VERSIONING__
|
|||
|
|
é )ÚTensorN)Ú
|
|||
|
|
functional)ÚAnyÚListÚOptionalÚTupleÚUnionÚDictÚSetÚCallable)%r ÚBaseImageProcessorÚBasePairwiseJudger ÚDatasetÚEvalPredictionÚFÚFeatureExtractionMixinÚIterableDatasetÚOnlineDPOTrainerÚOptimizerNamesr Ú PeftModelÚPreTrainedModelÚPreTrainedTokenizerBaseÚProcessorMixinÚSIMPLE_CHAT_TEMPLATEÚTrainerCallbackr Ú XPOConfigÚ
|
|||
|
|
XPOTrainerÚ empty_cacheÚgenerate_model_cardÚget_comet_experiment_urlÚ
|
|||
|
|
get_rewardÚis_conversationalÚis_peft_availableÚis_wandb_availableÚjinja2Úmaybe_apply_chat_templateÚnnÚosÚselective_log_softmaxÚtextwrapÚtorchÚtruncate_rightÚunwrap_model_for_generation)Ú*)Ú dataclassÚfield)ÚVersion)Ú nullcontext)ÚDataCollatorForSeq2SeqÚ DataCollatorForLanguageModelingTF)Úepilogue_fusionÚ max_autotuneÚ
|
|||
|
|
shape_paddingz
|
|||
|
|
trace.enabledztriton.cudagraphs)ÚdynamicÚ fullgraphÚoptionsc ó’ — t j | d| j d ¦ « dd¬¦ « }t j | d¦ « dd¬¦ « }g }t ||¦ « D ]‘\ }}| t j ¦ « }t j |d| d¦ « ¬¦ « d¦ « }t j
|
|||
|
|
|d¬¦ « }||z
|
|||
|
|
} | | ¦ « Œ’ t j |¦ « }| | j d | j d f¦ « }|S )Néÿÿÿÿé r )ÚchunksÚdim)r? Úindex©r? é )
|
|||
|
|
r+ ÚchunkÚreshapeÚshapeÚzipÚtoÚfloat32ÚgatherÚ unsqueezeÚsqueezeÚ logsumexpÚappendÚconcat)
|
|||
|
|
Úlogitsr@ Úchunked_logitsÚ
|
|||
|
|
chunked_indexÚall_per_token_logpsÚ chunk_logitsÚ chunk_indexÚselected_logitsÚlogsumexp_valuesÚper_token_logpss
|
|||
|
|
ú]/workspace/Fine-tuning/DS-LLM-TEMPLATE-FINETUNING/unsloth_compiled_cache/UnslothXPOTrainer.pyÚ chunked_selective_log_softmaxrY " s5 € õ ”[ § ¢ °°F´LÀÔ4DÑ!EÔ!EÐPQÐYZÐ[Ñ[Ô[€NÝ”[ § ¢ ¨rÑ!2Ô!2¸QÀaÐHÑHÔH€MØ Ðå%(¨ ¸ Ñ%GÔ%Gð 4ð 4Ñ!ˆ �kØ#—’¥u¤}Ñ5Ô5ˆ Ý œ, |¸2À{×G\ÒG\Ð]_ÑG`ÔG`ÐaÑaÔa×iÒiÐjlÑmÔmˆÝ œ?¨<¸rÐBÑBÔBÐØ)Ð,<Ñ<ˆØ×"Ò" ?Ñ3Ô3Ð3Ð3ØÝ œ,Ð':Ñ;Ô;ÐØ-×5Ò5°v´|ÀA´ÈÌ ÐUVÌÐ6XÑYÔYÐØ Ð ó c ó® ‡ — e Zd ZU dZ edddi¬¦ « Zee ed< edddi¬¦ « Z ee
|
|||
|
|
ed < d/ˆ fd.„ Z ˆ xZ S )0ÚUnslothXPOConfiga
|
|||
|
|
|
|||
|
|
Configuration class for the [`XPOTrainer`].
|
|||
|
|
|
|||
|
|
Subclass of [`OnlineDPOConfig`] we can use all its arguments and add the following:
|
|||
|
|
|
|||
|
|
Parameters:
|
|||
|
|
alpha (`float` or `list[float]`, *optional*, defaults to `1e-5`):
|
|||
|
|
Weight of the XPO loss term. If a list of floats is provided then the alpha is selected for each new epoch
|
|||
|
|
and the last alpha is used for the rest of the epochs.
|
|||
|
|
|
|||
|
|
NÚhelpzvLLM SamplingParams)ÚdefaultÚmetadataÚvllm_sampling_paramsr< z8Chunk size to reduce memory usage. -1 is most efficient.Úunsloth_num_chunksFÚnor= é r éú ç-C ëâ6
|
|||
|
|
?ç{®Gáz„?çÍÌÌÌÌÌì?ç+‡ÙÎ÷ï?ç:Œ0âŽyE>ç ð?ç @Úlinearçš™™™™™¹?ÚpassiveÚwarningTÚstepsrB éô éO
|
|||
|
|
ÚO1ÚautoÚ ç Ú
|
|||
|
|
adamw_8bitÚlengthÚ
|
|||
|
|
every_saveÚlasté é@ é ÚsigmoidÚvllmçš™™™™™á?c‘ óÌ •— |dk rt d|› d�¦ « ‚|dk rt d|› d�¦ « ‚|€|#dk r
|
|||
|
|
|$dk rd}d }#|ˆ€!d
|
|||
|
|
d lm}’ t |’¦ « d z d ¦ « }ˆ|…d
|
|||
|
|
k rt d
|
|||
|
|
¦ « ‚|…dk rt d¦ « ‚ t
|
|||
|
|
¦ « j dži d|“d|“d|“d|“d|“d|“d|“d|“d| “d|
|
|||
|
|
“d| “d| “d |
|
|||
|
|
“d |“d |“d |“d |“d!|“d"|“d#|“d$|“d%|“d&|“d'|“d(|“d)|“d*|“d+| “d,| “d-| “d.| “d/| “d0|!“d1|"“d2|#“d3|$“d4|%“d5|&“d6|'“d7|(“d8|)“d9|*“d:|+“d;|,“d<|-“d=|.“d>|/“d?|0“d@|1“dA|2“dB|3“dC|4“dD|5“dE|6“dF|7“dG|8“dH|9“dI|:“dJ|;“dK|<“dL|=“dM|>“dN|?“dO|@“dP|A“dQ|B“dR|C“dS|D“dT|E“dU|F“dV|G“dW|H“dX|I“dY|J“dZ|K“d[|L“d\|M“d]|N“d^|O“d_|P“d`|Q“da|R“db|S“dc|T“dd|U“de|V“df|W“dg|X“dh|Y“di|Z“dj|[“dk|\“dl|]“dm|^“dn|_“do|`“dp|a“dq|b“dr|c“ds|d“dt|e“du|f“dv|g“dw|h“dx|i“dy|j“dz|k“d{|l“d||m“d}|n“d~|o“d|p“d€|q“d�|r“d‚|s“dƒ|t“d„|u“d…|v“d†|w“d‡|x“dˆ|y“d‰|z“dŠ|{“d‹||“dŒ|}“d�|~“dŽ|“d�|€“d�|�“d‘|‚“d’|ƒ“d“|„“d”|…“d•|†“d–|‡“d—|ˆ“d˜|‰“d™|Š“dš|‹“d›|Œ“dœ|�“d�|Ž“|‘¤Ž |�| _ |�| _ d S )ŸNgH¯¼šò×z>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!rB za` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!rp rq Ú unsloth_training_checkpointsrb r )Ú cpu_countrc zUUnsloth: Please set a positive non-zero temperature since your results will be wrong.é
|
|||
|
|
zgUnsloth: Please set a positive non-zero temperature less than 10, since sampling will be quite erratic.Ú
|
|||
|
|
output_dirÚoverwrite_output_dirÚdo_trainÚdo_evalÚ
|
|||
|
|
do_predictÚ
|
|||
|
|
eval_strategyÚprediction_loss_onlyÚper_device_train_batch_sizeÚper_device_eval_batch_sizeÚper_gpu_train_batch_sizeÚper_gpu_eval_batch_sizeÚgradient_accumulation_stepsÚeval_accumulation_stepsÚ
|
|||
|
|
eval_delayÚtorch_empty_cache_stepsÚ
|
|||
|
|
learning_rateÚ weight_decayÚ
|
|||
|
|
adam_beta1Ú
|
|||
|
|
adam_beta2Ú adam_epsilonÚ
|
|||
|
|
max_grad_normÚnum_train_epochsÚ max_stepsÚlr_scheduler_typeÚ warmup_ratioÚ warmup_stepsÚ log_levelÚlog_level_replicaÚlog_on_each_nodeÚ logging_dirÚlogging_strategyÚlogging_first_stepÚ
|
|||
|
|
logging_stepsÚlogging_nan_inf_filterÚ
|
|||
|
|
save_strategyÚ
|
|||
|
|
save_stepsÚsave_total_limitÚsave_safetensorsÚsave_on_each_nodeÚsave_only_modelÚ'restore_callback_states_from_checkpointÚno_cudaÚuse_cpuÚuse_mps_deviceÚseedÚ data_seedÚ
|
|||
|
|
jit_mode_evalÚuse_ipexÚbf16Úfp16Úfp16_opt_levelÚhalf_precision_backendÚbf16_full_evalÚfp16_full_evalÚtf32Ú
|
|||
|
|
local_rankÚ ddp_backendÚ
|
|||
|
|
tpu_num_coresÚtpu_metrics_debugÚdebugÚdataloader_drop_lastÚ
|
|||
|
|
eval_stepsÚdataloader_num_workersÚdataloader_prefetch_factorÚ
|
|||
|
|
past_indexÚrun_nameÚ disable_tqdmÚremove_unused_columnsÚ label_namesÚload_best_model_at_endÚmetric_for_best_modelÚgreater_is_betterÚignore_data_skipÚfsdpÚfsdp_min_num_paramsÚ fsdp_configÚ"fsdp_transformer_layer_cls_to_wrapÚaccelerator_configÚ deepspeedÚlabel_smoothing_factorÚoptimÚ
|
|||
|
|
optim_argsÚ adafactorÚgroup_by_lengthÚlength_column_nameÚ report_toÚddp_find_unused_parametersÚddp_bucket_cap_mbÚddp_broadcast_buffersÚdataloader_pin_memoryÚ dataloader_persistent_workersÚskip_memory_metricsÚuse_legacy_prediction_loopÚ push_to_hubÚresume_from_checkpointÚ hub_model_idÚ hub_strategyÚ hub_tokenÚhub_private_repoÚhub_always_pushÚ hub_revisionÚgradient_checkpointingÚ gradient_checkpointing_kwargsÚinclude_inputs_for_metricsÚeval_do_concat_batchesÚ fp16_backendÚpush_to_hub_model_idÚpush_to_hub_organizationÚpush_to_hub_tokenÚ
|
|||
|
|
mp_parametersÚauto_find_batch_sizeÚfull_determinismÚ torchdynamoÚ ray_scopeÚ ddp_timeoutÚ
|
|||
|
|
torch_compileÚtorch_compile_backendÚtorch_compile_modeÚinclude_tokens_per_secondÚ include_num_input_tokens_seenÚneftune_noise_alphaÚoptim_target_modulesÚbatch_eval_metricsÚ
|
|||
|
|
eval_on_startÚuse_liger_kernelÚliger_kernel_configÚeval_use_gather_objectÚ average_tokens_across_devicesÚreward_model_pathÚjudgeÚmax_new_tokensÚ
|
|||
|
|
max_lengthÚ temperatureÚmissing_eos_penaltyÚ loss_typeÚdataset_num_procÚdisable_dropoutÚuse_vllmÚvllm_model_implÚgpu_memory_utilizationÚds3_gather_for_generationÚmodel_init_kwargs© )
|
|||
|
|
ÚFloatingPointErrorÚ
|
|||
|
|
OverflowErrorÚmultiprocessingrƒ ÚminÚ MathErrorÚsuperÚ__init__r` ra )”Úselfr… r† r‡ rˆ r‰ rŠ r‹ rŒ r� rŽ r� r� r‘ r’ r“ r” r• r– r— r˜ r™ rš r› rœ r� rž rŸ r r¡ r¢ r£ r¤ r¥ r¦ r§ r¨ r© rª r« r¬ r r® r¯ r° r± r² r³ r´ rµ r¶ r· r¸ r¹ rº r» r¼ r½ r¾ r¿ rÀ rÁ r rà rÄ rÅ rÆ rÇ rÈ rÉ rÊ rË rÌ rÍ rÎ rÏ rÐ rÑ rÒ rÓ rÔ rÕ rÖ r× rØ rÙ rÚ rÛ rÜ rÝ rÞ rß rà rá râ rã rä rå ræ rç rè ré rê rë rì rí rî rï rð rñ rò ró rô rõ rö r÷ rø rù rú rû rü rý rþ rÿ r r r r r r r r r r r
|
|||
|
|
r r r
|
|||
|
|
r r r r r r` ra Úkwargsrƒ Ú __class__s” €rX r zUnslothXPOConfig.__init__I s¿ ø€ ðh ˜4Ò Ð Õ'9ð ;VÐ]jð ;Vð ;Vð ;Vñ (Wô (Wð "WØ ˜1Ò Ð ¥Mð 3FÐUbð 3Fð 3Fð 3Fñ %Gô %Gð GØ Ð -°7Ò":Ð":¸zÈSÒ?PÐ?PØ7ˆJØ ˆMØ Ð #Ø 1Ð 1Ð 1Ð 1Ð 1Ð 1Ý " 9 9¡;¤;¨q¡=°!Ñ 4Ô 4Ð Ø ˜!Ò Ð ÝÐ sÑtÔtÐ tØ
|
|||
|
|
˜BÒ
|
|||
|
|
Ð
|
|||
|
|
Ýð Fñ Gô Gð
|
|||
|
|
Gð �‰ŒÔð N <ð N <ð N <Ø#˜ðN <à#7Ð#7ðN <ð �xðN <ð �gð N <ð
|
|||
|
|
$˜ð N <ð *˜Mð
|
|||
|
|
N <ð $8Ð#7ðN <ð +FÐ*EðN <ð *DÐ)CðN <ð (@Ð'?ðN <ð '>Ð&=ðN <ð +FÐ*EðN <ð '>Ð&=ðN <ð $˜ð N <ð '>Ð&=ð N <ð *˜Mð!N <ð" (˜<ð#N <ð$ $˜ð%N <ð& $˜ð'N <ð( (˜<ð)N <ð* *˜Mð+N <ð, 0Ð /ð-N <ð. "˜ ð/N <ð0 !2Ð 1ð1N <ð2 (˜<ð3N <ð4 (˜<ð5N <ð6 "˜ ð7N <ð8 !2Ð 1ð9N <ð: 0Ð /ð;N <ð< &˜+ð=N <ð> 0Ð /ð?N <ð@ "4Ð!3ðAN <ðB *˜MðCN <ðD &<Ð%;ðEN <ðF *˜MðGN <ðH $˜ðIN <ðJ 0Ð /ðKN <ðL 0Ð /ðMN <ðN !2Ð 1ðON <ðP .˜oðQN <ðR 7^Ð6]ðSN <ðT �gðUN <ðV �gðWN <ðX ,˜^ðYN <ðZ �4ð[N <ð\ "˜ ð]N <ð^ *˜Mð_N <ð` �xðaN <ðb �4ðcN <ðd �4ðeN <ðf ,˜^ðgN <ðh &<Ð%;ðiN <ðj ,˜^ðkN <ðl ,˜^ðmN <ðn �4ðoN <ðp $˜ðqN <ðr &˜+ðsN <ðt *˜MðuN <ðv !2Ð 1ðwN <ðx �EðyN <ðz $8Ð#7ð{N <ð| $˜ð}N <ð~ &<Ð%;ðN <ð@ *DÐ)CðAN <ðB $˜ðCN <ðD �xðEN <ðF (˜<ðGN <ðH %:Ð$9ðIN <ðJ &˜+ðKN <ðL &<Ð%;ðMN <ðN %:Ð$9ðON <ðP !2Ð 1ðQN <ðR 0Ð /ðSN <ðT �4ðUN <ðV #6Ð"5ðWN <ðX &˜+ðYN <ðZ 2TÐ1Sð[N <ð\ "4Ð!3ð]N <ð^ "˜ ð_N <ð` &<Ð%;ðaN <ðb �EðcN <ðd $˜ðeN <ðf "˜ ðgN <ðh .˜oðiN <ðj "4Ð!3ðkN <ðl "˜ ðmN <ðn *DÐ)CðoN <ðp !2Ð 1ðqN <ðr %:Ð$9ðsN <ðt %:Ð$9ðuN <ðv -JÐ,IðwN <ðx #6Ð"5ðyN <ðz *DÐ)Cð{N <ð| &˜+ð}N <ð~ &<Ð%;ðN <ð@ (˜<ðAN <ðB (˜<ðCN <ðD "˜ ðEN <ðF 0Ð /ðGN <ðH .˜oðIN <ðJ (˜<ðKN <ðL &<Ð%;ðMN <ðN -JÐ,IðON <ðP *DÐ)CðQN <ðR &<Ð%;ðSN <ðT (˜<ðUN <ðV $8Ð#7ðWN <ðX (@Ð'?ðYN <ðZ !2Ð 1ð[N <ð\ *˜Mð]N <ð^ $8Ð#7ð_N <ð` 0Ð /ðaN <ðb &˜+ðcN <ðd "˜ ðeN <ðf &˜+ðgN <ðh *˜MðiN <ðj %:Ð$9ðkN <ðl "4Ð!3ðmN <ðn )BÐ(AðoN <ðp -JÐ,IðqN <ðr #6Ð"5ðsN <ðt $8Ð#7ðuN <ðv "4Ð!3ðwN <ðx *˜MðyN <ðz 0Ð /ð{N <ð| #6Ð"5ð}N <ð~ &<Ð%;ðN <ð@ -JÐ,IðAN <ðB !2Ð 1ðCN <ðD �EðEN <ðF ,˜^ðGN <ðH $˜ðIN <ðJ &˜+ðKN <ðL #6Ð"5ðMN <ðN "˜ ðON <ðP 0Ð /ðQN <ðR .˜oðSN <ðT �xðUN <ðV .˜oðWN <ðX &<Ð%;ðYN <ðZ )BÐ(Að[N <ð\ !2Ð 1°Fð]N <ð N <ð N <ð^ %9ˆÔ!Ø"4ˆÔ Ð Ð rZ )�NNFFFrb Fr= r= NNrc rc r rd re rf rg rh ri rj rk r< rl rm r rn ro TNrp FrB Frp rq NTFFFFFFrr rr FFFFrs rt FFNr< NNFru FNr Nr< NNTNFNNFru r NNNNrv rw NFFrx NNNNTFTFFNNry NNFNFNFTrt NNNru TFNrz r{ FNNFFNNFFFNFTNNr| r} rg Nr~ NTFr r€ TNNr< )
|
|||
|
|
Ú__name__Ú
|
|||
|
|
__module__Ú __qualname__Ú__doc__r0 r` r r Ú__annotations__ra Úintr Ú
|
|||
|
|
__classcell__©r s @rX r\ r\ 3 sz ø€ € € € € € ð ð ð +0¨%ØØÐ 1Ð2ð+ñ +ô +И( 3œ-ð ð ñ ð */¨ØØÐ VÐWð*ñ *ô *И #œ ð ð ñ ð Ø #ØØØØ Ø $Ø&'Ø%&Ø#'Ø"&Ø&'Ø"#ØØ"%Ø ØØØØ ØØ ØØ $ØØØ Ø %Ø ØØ"Ø "ØØ!&Ø ØØ Ø Ø !Ø Ø27ØØØ ØØØ ØØØØ Ø!'Ø Ø ØØØØ Ø !ØØ $ØØ!"Ø%)ØØØØ $ØØ!&Ø $Ø Ø ØØ ØØ-1Ø !ØØ!$Ø ØØØ Ø %ØØ%)Ø Ø $Ø $Ø(-Ø "Ø%*ØØ!%ØØ#ØØ Ø ØØ!&Ø(,Ø%*Ø!%Ø Ø #Ø#'Ø ØØ #Ø ØØØØ Ø $Ø !Ø$)Ø(-Ø "Ø #Ø "Ø Ø Ø "Ø!&Ø(,Ø ØØØØØ "Ø Ø Ø ØØ Ø!%Ø$(Ø Ø #Ø ðcr5ð r5ð r5ð r5ð r5ð r5ð r5ð r5ð r5ð r5rZ r\ c ó ‡ — e Zd ZdZddgZ d%deeej f deeej f de ej d e e
|
|||
|
|
d
|
|||
|
|
e e d e e d e ee
|
|||
|
|
ef d
|
|||
|
|
e ee
|
|||
|
|
eee
|
|||
|
|
f f de eeeeef de e de e egef de ee deej j ej j j f de e ej ej gej f ddf ˆ fd„
|
|||
|
|
Z e d„ ¦ « Z!d„ Z"d„ Z#d„ Z$d„ Z%d„ Z&d „ Z' dd „Z( d&dej d eeeej e)f f d e e* dej fd „Z+ d'd!e e d"e e d#eeee df fd$„Z,ˆ xZ-S )(Ú_UnslothXPOTrainerru ÚtrlÚxpoN©NNÚmodelÚ ref_modelÚ reward_modelr ÚargsÚ
|
|||
|
|
data_collatorÚ
|
|||
|
|
train_datasetÚ eval_datasetÚprocessing_classÚ peft_configÚcompute_metricsÚ callbacksÚ
|
|||
|
|
optimizersÚ preprocess_logits_for_metricsÚreturnc ó •— t ¦ « ||||||||| | |
|
|||
|
|
| | |
|
|||
|
|
|¬¦ « | j j | _ g g g g g g g g g g g g g g dœ| _ | j � g | j d< g | j d< g | j d< d S d S )N)r+ r, r r- r. r/ r0 r1 r2 Úreward_processing_classr3 r4 r5 r6 r7 )úloss/dpoúloss/xpoú objective/klúobjective/entropyúrewards/chosenúrewards/rejectedúrewards/accuraciesúrewards/marginsú logps/chosenúlogps/rejectedúval/model_contain_eos_tokenúval/ref_contain_eos_tokenÚalphaÚbetaúobjective/model_scoresúobjective/ref_scoresúobjective/scores_margin)r r r. rG Ú_alphaÚstatsr- )r r+ r, r- r r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r s €rX r z_UnslothXPOTrainer.__init__ƒ sÚ ø€ õ& ‰Œ×ÒØØ ØØ%ØØ'Ø'Ø%Ø -Ø$4Ø#Ø +Ø Ø!Ø*Gð ñ
|
|||
|
|
ô
|
|||
|
|
ð
|
|||
|
|
ð$ ”i”oˆŒ ð ØØ Ø!#Ø Ø "Ø"$Ø !Ø Ø à+-Ø)+ØØð#
|
|||
|
|
ð
|
|||
|
|
ˆŒ
|
|||
|
|
ð& Ô Ð (à35ˆDŒJÐ/Ñ 0Ø13ˆDŒJÐ-Ñ .Ø46ˆDŒJÐ0Ñ 1Ð 1Ð 1ð )Ð (rZ c óÀ — t | j t ¦ « r>| j j }|t | j ¦ « k r
|
|||
|
|
| j | n | j d S | j S )Nr< )Ú
|
|||
|
|
isinstancerL ÚlistÚstateÚepochÚlen)r rR s rX rG z_UnslothXPOTrainer.alphaÄ sU € å �d”k¥4Ñ (Ô (ð Ø”JÔ$ˆEØ).µ°T´[Ñ1AÔ1AÒ)AÐ)A�4”;˜uÔ%Ð%ÀtÄ{ÐSUÄÐ Và”;Ð rZ c óX — t || j ¦ « 5 }| |d |d | j ¬¦ « }d d d ¦ « n # 1 swxY w Y | j €U| j |¦ « }t
|
|||
|
|
¦ « r*t |t ¦ « r| ¦ « }n"|}n | j | j ¦ « }t || j ¦ « 5 }| |d |d | j ¬¦ « }d d d ¦ « n # 1 swxY w Y ||fS )NÚ input_idsÚattention_mask)rU rV Úgeneration_config)
|
|||
|
|
r- Ú acceleratorÚgeneraterW r, Ú unwrap_modelr# rO r Úget_base_model) r Úpromptsr+ Ú unwrapped_policy_model_for_genÚ model_outputÚ"unwrapped_main_model_for_ref_logicÚ actual_model_for_ref_generationÚfinal_ref_model_for_genÚ
|
|||
|
|
ref_outputs rX Ú_generate_completionsz(_UnslothXPOTrainer._generate_completionsÌ sº € Ý
|
|||
|
|
(¨°Ô0@Ñ
|
|||
|
|
AÔ
|
|||
|
|
Að ÐEcØ9×BÒBØ! +Ô.Ø &Ð'7Ô 8Ø"&Ô"8ð Cñ ô ˆLð ð ð ñ ô ð ð ð ð ð ð øøøð ð ð ð ð Œ>Ð !Ø15Ô1A×1NÒ1NÈuÑ1UÔ1UÐ .å Ñ"Ô"ð
|
|||
|
|
U¥zÐ2TÕV_Ñ'`Ô'`ð
|
|||
|
|
UØ2T×2cÒ2cÑ2eÔ2eÐ/Ð/à2TÐ/Ð/à.2Ô.>×.KÒ.KÈDÌNÑ.[Ô.[Ð +å
|
|||
|
|
(Ð)HÈ$ÔJZÑ
|
|||
|
|
[Ô
|
|||
|
|
[ð Ð_vØ0×9Ò9Ø! +Ô.Ø &Ð'7Ô 8Ø"&Ô"8ð :ñ ô ˆJð ð ð ñ ô ð ð ð ð ð ð øøøð ð ð ð ð ˜ZÐ'Ð's# –*A Á AÁAÃ'*D Ä D!Ä$D!c ó — |d j d }|d d …|d …f }t || j j | j j ¦ « \ }}t j |d |fd¬¦ « t j |d |fd¬¦ « |d dœ}|d d …|d …f }t || j j | j j ¦ « \ }} t j |d |fd¬¦ « t j |d | fd¬¦ « |d dœ}
|
|||
|
|
||
|
|||
|
|
fS )NrU rB rA rV Úraw©rU rV re )rE r, r2 Ú eos_token_idÚ pad_token_idr+ Úcat) r r^ rb r\ Úcontext_lengthÚmodel_completion_idsÚmodel_completion_maskÚ
|
|||
|
|
model_dataÚref_completion_idsÚref_completion_maskÚref_datas rX Ú_process_completionsz'_UnslothXPOTrainer._process_completionsè sK € Ø Ô-Ô3°AÔ6ˆð ,¨A¨A¨A¨~¨¨Ð,>Ô ?Ð Ý6DØ $Ô"7Ô"DÀdÔF[ÔFhñ7
|
|||
|
|
ô 7
|
|||
|
|
Ñ3Ð Ð 3õ œ G¨KÔ$8Ð:NÐ#OÐUVÐWÑWÔWÝ #œi¨Ð1AÔ)BÐDYÐ(ZÐ`aÐ bÑ bÔ bؘ5”>ð
|
|||
|
|
ð
|
|||
|
|
ˆ
|
|||
|
|
ð (¨¨¨¨>¨?¨?Ð(:Ô ;ÐÝ2@Ø Ô 5Ô BÀDÔDYÔDfñ3
|
|||
|
|
ô 3
|
|||
|
|
Ñ/ÐÐ /õ œ G¨KÔ$8Ð:LÐ#MÐSTÐUÑUÔUÝ #œi¨Ð1AÔ)BÐDWÐ(XÐ^_Ð `Ñ `Ô `ؘ5”>ð
|
|||
|
|
ð
|
|||
|
|
ˆð ˜8Ð#Ð#rZ c ó8 — t j ¦ « 5 t | j |d | j j |¦ « \ }}}t | j |d | j j |¦ « \ }}}d d d ¦ « n # 1 swxY w Y | j j �Št j |d | j j k d¬¦ « }t j |d | j j k d¬¦ « }|| xx | j j z cc< || xx | j j z cc< ||fS )NrU r< rA )
|
|||
|
|
r+ Úno_gradr! r- r2 rh r. r
|
|||
|
|
Úanyrg ) r rm rp rj Ú_Ú model_scoresÚ
|
|||
|
|
ref_scoresÚmodel_contain_eosÚref_contain_eoss rX Ú_compute_rewardsz#_UnslothXPOTrainer._compute_rewards sm € Ý
|
|||
|
|
Œ]‰_Œ_ð ð Ý!+ØÔ! :¨kÔ#:¸DÔ<QÔ<^Ð`nñ"ô "Ñ ˆAˆ|˜Qõ *ØÔ! 8¨KÔ#8¸$Ô:OÔ:\Ð^lñ ô Ñ ˆAˆz˜1ð ð ð ñ ô ð ð ð ð ð ð øøøð ð ð ð ð Œ9Ô (Ð 4Ý %¤ ¨*°[Ô*AÀTÔEZÔEgÒ*gÐmoÐ pÑ pÔ pÐ Ý #œi¨°Ô(=ÀÔAVÔAcÒ(cÐikÐ lÑ lÔ lˆOØ Ð+Ð+Ð ,Ð ,Ô ,°´ Ô0MÑ MÐ ,Ð ,Ñ ,Ø ˜Ð'Ð (Ð (Ô (¨D¬IÔ,IÑ IÐ (Ð (Ñ (à˜ZÐ'Ð's ”AA7Á7A;Á>A;c óÊ ‡ — |d }| j |d d d …|d …f d¬¦ « }d„ |D ¦ « }| j |d d d …|d …f d¬¦ « }d„ |D ¦ « }t d|d i¦ « rod „ |D ¦ « }t j ¦ « }| t ¦ « Š ˆ fd
|
|||
|
|
„|D ¦ « }ˆ fd „|D ¦ « }d „ |D ¦ « }ˆ fd
|
|||
|
|
„|D ¦ « }| j |t t ||¦ « ¦ « ¦ « }t j d„ |D ¦ « |d j ¬¦ « S )Nre rU T)Úskip_special_tokensc ó6 — g | ]}| ¦ « ‘ŒS r ©Ústrip©Ú.0Ú
|
|||
|
|
completions rX ú
|
|||
|
|
<listcomp>z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp> s$ € Ð!^Ð!^Ð!^¸ *×"2Ò"2Ñ"4Ô"4Ð!^Ð!^Ð!^rZ c ó6 — g | ]}| ¦ « ‘ŒS r r~ r€ s rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp> s$ € Ð ZÐ ZÐ Z°z
|
|||
|
|
× 0Ò 0Ñ 2Ô 2Ð ZÐ ZÐ ZrZ Úpromptr c ó — g | ]}d |dœg‘Œ S ©Ú assistant)ÚroleÚcontentr r€ s rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>" s0 € ð &ð &ð &ØCM˜+°*Ð=Ð=Ð>ð&ð &ð &rZ c ó< •— g | ]}‰ |¬ ¦ « ‘ŒS ©)Úmessages©Úrender)r� ÚmessageÚtemplates €rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>' s' ø€ ÐPÐPÐP¸W�x—’°�Ñ8Ô8ÐPÐPÐPrZ c ó< •— g | ]}‰ |¬ ¦ « ‘ŒS rŒ rŽ ©r� r‚ r‘ s €rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>( s' ø€ Ð%tÐ%tÐ%tÈz h§o¢o¸z oÑ&JÔ&JÐ%tÐ%tÐ%trZ c ó — g | ]}d |dœg‘Œ S r‡ r r€ s rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>* s0 € ð $ð $ð $ØCM˜+°*Ð=Ð=Ð>ð$ð $ð $rZ c ó< •— g | ]}‰ |¬ ¦ « ‘ŒS rŒ rŽ r“ s €rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>- s' ø€ Ð#pÐ#pÐ#pÈZ H§O¢O¸Z OÑ$HÔ$HÐ#pÐ#pÐ#prZ c ó — g | ]}|d k ‘Œ S )r r )r� Úranks rX rƒ z5_UnslothXPOTrainer._compute_judge.<locals>.<listcomp>6 s € Ð MÐ MÐ M¨4˜T QšYÐ MÐ MÐ MrZ )Údevice)
|
|||
|
|
r2 Ú batch_decoder" r% Ú EnvironmentÚ from_stringr r rP rF r+ Útensorr˜ )
|
|||
|
|
r rm rp rj r\ Úmodel_data_completionsÚref_data_completionsÚ environmentÚranks_of_first_completionr‘ s
|
|||
|
|
@rX Ú_compute_judgez!_UnslothXPOTrainer._compute_judge sä ø€ Ø ˜UÔ#ˆØ!%Ô!6×!CÒ!CØ �{Ô # A A A ~ Ð$6Ô 7ÈTð "Dñ "
|
|||
|
|
ô "
|
|||
|
|
Ð ð "_Ð!^ÐG]Ð!^Ñ!^Ô!^Ð à #Ô 4× AÒ AØ �[Ô ! ! ! ! ^ _ _Ð"4Ô 5È4ð Bñ
|
|||
|
|
ô
|
|||
|
|
Ð ð [Ð ZÐEYÐ ZÑ ZÔ ZÐ å ˜h¨°¬
|
|||
|
|
Ð 3Ñ 4Ô 4ð qð&ð &ØQgð&ñ &ô &Ð "õ !Ô,Ñ.Ô.ˆKØ"×.Ò.Õ/CÑDÔDˆHØPÐPÐPÐPÈÐPÑPÔPˆGØ%tÐ%tÐ%tÐ%tÐ]sÐ%tÑ%tÔ%tÐ "ð$ð $ØQeð$ñ $ô $Ð ð $qÐ#pÐ#pÐ#pÐ[oÐ#pÑ#pÔ#pÐ à$(¤J×$4Ò$4Ø Ý •Ð+Ð-AÑBÔBÑ CÔ Cñ%
|
|||
|
|
ô %
|
|||
|
|
Ð!õ Œ|Ð MÐ MÐ3LÐ MÑ MÔ MÐV`ÐalÔVmÔVtÐuÑuÔuÐurZ c óŒ ‡— ˆfd„} |||¦ « } |||¦ « }t j ¦ « 5 | j €E| ¦ « 5 |||¦ « } |||¦ « } d d d ¦ « n # 1 swxY w Y n" || j |¦ « } || j |¦ « } d d d ¦ « n # 1 swxY w Y |d d d …‰d …f dk }
|
|||
|
|
|d d d …‰d …f dk } | |
|
|||
|
|
d¦ « }| | d¦ « }| | d¦ « } | |
|
|||
|
|
d¦ « }||| |fS )Nc óª •— | |d |d ¬¦ « }|j d d …‰dz
|
|||
|
|
d…f }t ||d d d …‰d …f ¦ « }|S )NrU rV )rV rB r< )rO r) )ÚmÚdataÚoutputrO Útoken_logprobsrj s €rX Úcompute_logprobs_for_datazG_UnslothXPOTrainer._compute_logprobs.<locals>.compute_logprobs_for_data9 sp ø€ Ø�Q�t˜KÔ(¸Ð>NÔ9OÐPÑPÔPˆFØ”] 1 1 1 n°qÑ&8¸2Ð&=Ð#=Ô>ˆFÝ 2°6¸4À Ô;LÈQÈQÈQÐP^ÐP_ÐP_ÐM_Ô;`Ñ aÔ aˆNØ!Ð !rZ rV r rv )r+ rs r, Údisable_adapterÚ masked_fill) r r+ rm rp rj r¨ Úmodel_logprobs_model_dataÚmodel_logprobs_ref_dataÚref_logprobs_model_dataÚref_logprobs_ref_dataÚmodel_padding_maskÚref_padding_masks ` rX Ú_compute_logprobsz$_UnslothXPOTrainer._compute_logprobs8 sI ø€ ð "ð "ð "ð "ð "ð %>Ð$=¸eÀZÑ$PÔ$PÐ!à";Ð";¸EÀ8Ñ"LÔ"LÐ õ Œ]‰_Œ_ð \ð \ØŒ~Ð%Ø×*Ò*Ñ,Ô,ð Wð WØ.GÐ.GÈÈzÑ.ZÔ.ZÐ+Ø,EÐ,EÀeÈXÑ,VÔ,VÐ)ðWð Wð Wñ Wô Wð Wð Wð Wð Wð Wð Wøøøð Wð Wð Wð Wøð +DÐ*CÀDÄNÐT^Ñ*_Ô*_Ð'Ø(AÐ(AÀ$Ä.ÐRZÑ([Ô([Ð%ð \ð \ð \ñ \ô \ð \ð \ð \ð \ð \ð \øøøð \ð \ð \ð \ð (Ð(8Ô 9¸!¸!¸!¸^¸_¸_Ð:LÔ MÐQRÒ RÐØ#Ð$4Ô5°a°a°a¸ ¸ ¸ Ð6HÔIÈQÒNÐØ$=×$IÒ$IÐJ\Ð^aÑ$bÔ$bÐ!Ø"9×"EÒ"EÐFVÐX[Ñ"\Ô"\Ð Ø 5× AÒ AÐBRÐTWÑ XÔ XÐ Ø"9×"EÒ"EÐFXÐZ]Ñ"^Ô"^Ð à(Ð*AÐCXÐZqÐqÐqs5 ² B-ÁA3Á' B-Á3A7 Á7B-Á:A7 Á;&B-Â-B1Â4B1c ó¤ — | d¦ « }| d¦ « }| d¦ « }| d¦ « } t j |||¦ « }
|
|||
|
|
t j || |¦ « } |
|
|||
|
|
| z
|
|||
|
|
} t j | ||¦ « }
|
|||
|
|
t j | | |¦ « }|
|
|||
|
|
|z
|
|||
|
|
}| |z
|
|||
|
|
}| j j dk r t j | j |z ¦ « }n@| j j dk r|dd| j z z z
|
|||
|
|
dz }n t d| j j › �¦ « ‚| j |z }||z
|
|||
|
|
¦ « }|||fS )NrB r~ Úiporc zinvalid loss type ) Úsumr+ Úwherer. r r Ú
|
|||
|
|
logsigmoidrH ÚNotImplementedErrorrG Úmean)r r« r¬ r® r Ú chosen_maskÚ model_logprobs_model_data_sumÚmodel_logprobs_ref_data_sumÚref_logprobs_ref_data_sumÚref_logprobs_model_data_sumÚchosen_model_logprobsÚchosen_ref_logprobsÚchosen_log_ratiosÚrejected_model_logprobsÚrejected_ref_logprobsÚrejected_log_ratiosrO Ú
|
|||
|
|
dpo_lossesÚ
|
|||
|
|
xpo_lossesÚlosss rX Ú_compute_lossesz"_UnslothXPOTrainer._compute_lossesX s} € ð )B×(EÒ(EÀaÑ(HÔ(HÐ%Ø&=×&AÒ&AÀ!Ñ&DÔ&DÐ#Ø$9×$=Ò$=¸aÑ$@Ô$@Ð!Ø&=×&AÒ&AÀ!Ñ&DÔ&DÐ#å %¤ ¨KÐ9VÐXsÑ tÔ tÐ Ý #œk¨+Ð7RÐTmÑ nÔ nÐØ 1Ð4GÑ GÐå"'¤+¨{¨lÐ<YÐ[vÑ"wÔ"wÐ Ý %¤ ¨[¨LÐ:UÐWpÑ qÔ qÐ Ø 5Ð8MÑ MÐð #Ð%8Ñ8ˆà Œ9Ô )Ò +Ð +Ýœ, t¤y°6Ñ'9Ñ:Ô:Ð:ˆJˆJØ
|
|||
|
|
ŒYÔ
|
|||
|
|
EÒ
|
|||
|
|
)Ð
|
|||
|
|
)Ø 1¨¨D¬I©
|
|||
|
|
Ñ#6Ñ6¸1Ñ<ˆJˆJå%Ð&P¸4¼9Ô;NÐ&PÐ&PÑQÔQÐ Qð ”ZÐ"=Ñ=ˆ
|
|||
|
|
ð ˜ZÑ'×-Ò-Ñ/Ô/ˆà�Z Ð+Ð+rZ c
|
|||
|
|
óê
|
|||
|
|
‡ — ˆ fd„}
|
|||
|
|
‰ j d |
|
|||
|
|
|¦ « ¦ « ‰ j d |
|
|||
|
|
| ¦ « ¦ « ‰ j �~‰ j d |
|
|||
|
|
| ¦ « ¦ « ‰ j d |
|
|||
|
|
| ¦ « ¦ « ‰ j d |
|
|||
|
|
| | z
|
|||
|
|
¦ « ¦ « | d¦ « }| d¦ « }| d¦ « }| d¦ « }t j |||¦ « }t j |||¦ « }||z
|
|||
|
|
}t j | ||¦ « }t j | ||¦ « }||z
|
|||
|
|
}‰ j d |
|
|||
|
|
| ¦ « | ¦ « z ¦ « ¦ « ‰ j d |
|
|||
|
|
| ¦ « | ¦ « z ¦ « ¦ « |‰ j z }|‰ j z }‰ j d
|
|||
|
|
|
|
|||
|
|
| ¦ « ¦ « ¦ « ‰ j d |
|
|||
|
|
| ¦ « ¦ « ¦ « ||z
|
|||
|
|
}||z
|
|||
|
|
}| d¦ « | d¦ « z ¦ « d z } ‰ j d
|
|||
|
|
|
|
|||
|
|
| ¦ « ¦ « | d¦ « } | d¦ « } | ¦ « | ¦ « z d z } ‰ j d |
|
|||
|
|
| ¦ « ¦ « ||z
|
|||
|
|
} ‰ j d |
|
|||
|
|
| ¦ « ¦ « ¦ « | dk ¦ « }!‰ j d |
|
|||
|
|
|! ¦ « ¦ « ¦ « |d d d …|
|
|||
|
|
d …f ‰ j j
|
|||
|
|
k d¬¦ « }"|d d d …|
|
|||
|
|
d …f ‰ j j
|
|||
|
|
k d¬¦ « }#‰ j d |
|
|||
|
|
|" ¦ « ¦ « ¦ « ‰ j d |
|
|||
|
|
|# ¦ « ¦ « ¦ « ‰ j d ‰ j ¦ « ‰ j d ‰ j ¦ « d S )Nc ó€ •— ‰j | ¦ « ¦ « ¦ « S ©N)rX Úgather_for_metricsr¸ Úitem)rœ r s €rX Ú gather_meanz7_UnslothXPOTrainer._log_statistics.<locals>.gather_mean� s2 ø€ ØÔ#×6Ò6°vÑ>Ô>×CÒCÑEÔE×JÒJÑLÔLÐ LrZ r; r< rI rJ rK rB rC rD r? r@ rc r= r> rB r rA rU rA rE rF rG rH )
|
|||
|
|
rM rM r- r´ r+ rµ r¸ rH Úfloatr2 rg rt rG )$r rm rp r« r¬ r® r r¹ rÄ rÅ rj rv rw rÍ rº r» r¼ r½ r¾ r¿ rÀ rÁ r rà Úchosen_rewardsÚrejected_rewardsÚ
|
|||
|
|
kl_model_dataÚ kl_ref_dataÚmean_klÚentropy_model_dataÚentropy_ref_dataÚ mean_entropyÚmarginÚaccuracyÚ model_eosÚref_eoss$ ` rX Ú_log_statisticsz"_UnslothXPOTrainer._log_statistics€ sD ø€ ð Mð Mð Mð Mð Mð
|
|||
|
|
Œ
|
|||
|
|
�:Ô ×%Ò% k k°*Ñ&=Ô&=Ñ>Ô>Ð>Ø Œ
|
|||
|
|
�:Ô ×%Ò% k k°*Ñ&=Ô&=Ñ>Ô>Ð>ð Ô Ð (Ø ŒJÐ/Ô 0× 7Ò 7¸ ¸ ÀLÑ8QÔ8QÑ RÔ RÐ RØ ŒJÐ-Ô .× 5Ò 5°k°kÀ*Ñ6MÔ6MÑ NÔ NÐ NØ ŒJÐ0Ô 1× 8Ò 8¸¸À\ÐT^ÑE^Ñ9_Ô9_Ñ `Ô `Ð `ð )B×(EÒ(EÀaÑ(HÔ(HÐ%Ø&=×&AÒ&AÀ!Ñ&DÔ&DÐ#Ø$9×$=Ò$=¸aÑ$@Ô$@Ð!Ø&=×&AÒ&AÀ!Ñ&DÔ&DÐ#å %¤ ¨KÐ9VÐXsÑ tÔ tÐ Ý #œk¨+Ð7RÐTmÑ nÔ nÐØ 1Ð4GÑ GÐå"'¤+¨{¨lÐ<YÐ[vÑ"wÔ"wÐ Ý %¤ ¨[¨LÐ:UÐWpÑ qÔ qÐ Ø 5Ð8MÑ MÐà Œ
|
|||
|
|
�>Ô"×)Ò)¨+¨+Ð6K×6PÒ6PÑ6RÔ6RÐUh×UmÒUmÑUoÔUoÑ6oÑ*pÔ*pÑqÔqÐqØ Œ
|
|||
|
|
Ð#Ô$×+Ò+¨K¨KÐ8O×8TÒ8TÑ8VÔ8VÐYn×YsÒYsÑYuÔYuÑ8uÑ,vÔ,vÑwÔwÐwð +¨T¬YÑ6ˆØ.°´Ñ:ÐØ Œ
|
|||
|
|
Ð#Ô$×+Ò+¨K¨K¸×8KÒ8KÑ8MÔ8MÑ,NÔ,NÑOÔOÐOØ Œ
|
|||
|
|
Ð%Ô&×-Ò-¨k¨kÐ:J×:OÒ:OÑ:QÔ:QÑ.RÔ.RÑSÔSÐSð 2Ð4KÑKˆ
|
|||
|
|
Ø-Ð0EÑEˆ Ø ×$Ò$ QÑ'Ô'¨+¯/ª/¸!Ñ*<Ô*<Ñ<×BÒBÑDÔDÀqÑHˆØ Œ
|
|||
|
|
�>Ô"×)Ò)¨+¨+°gÑ*>Ô*>Ñ?Ô?Ð?ð 8× ;Ò ;¸AÑ >Ô >Ð >ÐØ 3× 7Ò 7¸Ñ :Ô :Ð:ÐØ*×/Ò/Ñ1Ô1Ð4D×4IÒ4IÑ4KÔ4KÑKÈqÑPˆ Ø Œ
|
|||
|
|
Ð&Ô'×.Ò.¨{¨{¸<Ñ/HÔ/HÑIÔIÐIð Ð"2Ñ2ˆØ Œ
|
|||
|
|
Ð$Ô%×,Ò,¨[¨[¸¿º¹ ¼ Ñ-GÔ-GÑHÔHÐHð ˜Q’J×%Ò%Ñ'Ô'ˆØ Œ
|
|||
|
|
Ð'Ô(×/Ò/° ° ¸H¿MºM¹O¼OÑ0LÔ0LÑMÔMÐMð Ô,¨Q¨Q¨Q°°°Ð-?Ô@ÀDÔDYÔDfÒf×kÒkÐpqÐkÑrÔrˆ ؘKÔ(¨¨¨¨N¨O¨OÐ);Ô<ÀÔ@UÔ@bÒb×gÒgÐlmÐgÑnÔnˆØ Œ
|
|||
|
|
Ð0Ô1×8Ò8¸¸ÀYÇ_Â_ÑEVÔEVÑ9WÔ9WÑXÔXÐXØ Œ
|
|||
|
|
Ð.Ô/×6Ò6°{°{À7Ç=Â=Á?Ä?Ñ7SÔ7SÑTÔTÐTð
|
|||
|
|
Œ
|
|||
|
|
�7Ô×"Ò" 4¤:Ñ.Ô.Ð.Ø Œ
|
|||
|
|
�6Ô×!Ò! $¤)Ñ,Ô,Ð,Ð,Ð,rZ ÚinputsÚnum_items_in_batchc ó4 ‡ ‡— | ¦ « t t t ‰ ¦ « ¦ « ¦ « ¦ « }‰d }ˆfd„t |¦ « D ¦ « Šˆ fd„‰D ¦ « Šˆ fd„‰D ¦ « Š‰ ‰¦ « Š‰ ‰¦ « Š‰d j d }‰d ‰d |dœ}‹‰ ||¦ « \ }}‰
|
|||
|
|
|||¦ « \ } }
|
|||
|
|
‰ j �!‰ | |
|
|||
|
|
|¦ « \ } } | | k }
|
|||
|
|
n d \ } } ‰
|
|||
|
|
| |
|
|||
|
|
|¦ « }
|
|||
|
|
‰ || |
|
|||
|
|
|¦ « \ }}}}‰ |||||
|
|||
|
|
¦ « \ }}}‰ | |
|
|||
|
|
| ¦ « | ¦ « |||
|
|||
|
|
| ¦ « | ¦ « || | ¦ « ‰ j j �+‰ j j ‰ j j z d
|
|||
|
|
k rt- ¦ « i }‰ j j t0 j t0 j fv r‰ ¦ « |d < ‰ j j dk r| ¦ « }‰ j rMt> |‰ j! ¦ « 5 }| " ¦ « d d d ¦ « n # 1 swxY w Y n ‰ j# j" |fi |¤Ž | ¦ « ‰ j j$ z S ) Nr… c óR •‡— g | ]"Šˆfd „‰ ¦ « D ¦ « ‘Œ#S )c ó( •— i | ]\ }}||‰ “ŒS r r )r� ÚkÚvÚis €rX ú
|
|||
|
|
<dictcomp>z?_UnslothXPOTrainer.training_step.<locals>.<listcomp>.<dictcomp>Û s# ø€ Ð6Ð6Ð6™t˜q !�1�a˜”dÐ6Ð6Ð6rZ )Úitems)r� rã rÜ s @€rX rƒ z4_UnslothXPOTrainer.training_step.<locals>.<listcomp>Û s7 øø€ ÐRÐRÐR¸1Ð6Ð6Ð6Ð6 v§|¢|¡~¤~Ð6Ñ6Ô6ÐRÐRÐRrZ c ó: •— g | ]}t |‰j ¦ « ‘ŒS r )r& r2 ©r� Úxr s €rX rƒ z4_UnslothXPOTrainer.training_step.<locals>.<listcomp>Ü s' ø€ ÐVÐVÐVÈ!Õ+¨A¨tÔ/DÑEÔEÐVÐVÐVrZ c óf •— g | ]-}‰ |‰j j j ‰j ¦ « ‘Œ.S r )Ú tokenize_rowr+ ÚconfigÚis_encoder_decoderr2 rç s €rX rƒ z4_UnslothXPOTrainer.training_step.<locals>.<listcomp>Ý s7 ø€ ÐtÐtÐtÐhi�$×#Ò# A t¤zÔ'8Ô'KÈTÔMbÑcÔcÐtÐtÐtrZ Úprompt_input_idsrB Úprompt_attention_maskrf r* r r” )%ÚtrainrS ÚnextÚiterÚvaluesÚranger/ Ú_prepare_inputsrE rc rq r- rz r¡ r± rÇ rÛ Údetachr. r“ rQ Ú global_stepr rÕ r ÚLOMOÚADALOMOÚ_get_learning_rateÚn_gpur¸ Úuse_apexÚampÚ
|
|||
|
|
scale_lossÚ optimizerÚbackwardrX r� )r r+ rÜ rÝ Ú
|
|||
|
|
batch_sizer\ rj r^ rb rm rp rv rw r¹ r« r¬ r® r rÆ rÄ rÅ r Ú scaled_losss ` ` rX Ú
|
|||
|
|
training_stepz _UnslothXPOTrainer.training_stepÓ s� øø€ ð � Š ‰
|
|||
|
|
Œ
|
|||
|
|
ˆ
|
|||
|
|
õ ��d 6§=¢=¡?¤?Ñ 3Ô 3Ñ4Ô4Ñ5Ô5ˆ
|
|||
|
|
ؘÔ"ˆØRÐRÐRÐRÅÀjÑ@QÔ@QÐRÑRÔRˆØVÐVÐVÐVÈvÐVÑVÔVˆØtÐtÐtÐtÐmsÐtÑtÔtˆØ×#Ò# FÑ+Ô+ˆð ×%Ò% fÑ-Ô-ˆØ Ð 2Ô3Ô9¸!Ô<ˆà Ð 2Ô3Ø $Ð%<Ô =Øð
|
|||
|
|
ð
|
|||
|
|
ˆð
|