250 lines
38 KiB
Plaintext
250 lines
38 KiB
Plaintext
|
§
|
|||
|
|
5$�hÞ› ã ó — d Z ddlmZ ddlZddlmZ ddlmZ ddlmZm Z m
|
|||
|
|
Z
|
|||
|
|
m Z m Z m
|
|||
|
|
Z
|
|||
|
|
mZmZ ddlmZmZmZmZmZmZmZm
|
|||
|
|
Z
|
|||
|
|
mZmZmZmZmZm Z m Z m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(mZm)Z)m*Z*m+Z+mZm,Z,m
|
|||
|
|
Z
|
|||
|
|
mZm Z m Z m'Z'm)Z)mZ ddl)Z)ddlT ddl-m.Z.m/Z/ dd l0m1Z1 ddlZddl2Z3dd
|
|||
|
|
l4m5Z5 ddlmZ dd l6m7Z7m8Z9 d d
|
|||
|
|
d d
|
|||
|
|
d
|
|||
|
|
dœZ: ej; d d e:¬¦ « d„ ¦ « Z<e. G d„ de¦ « ¦ « Z= G d„ de ¦ « Z> G d„ de>¦ « Z?dS )z8
|
|||
|
|
2025.8.4
|
|||
|
|
2025.8.5
|
|||
|
|
4.55.1
|
|||
|
|
0.21.0
|
|||
|
|
__UNSLOTH_VERSIONING__
|
|||
|
|
é )ÚTensorN)Ú
|
|||
|
|
functional)ÚAnyÚListÚOptionalÚTupleÚUnionÚDictÚSetÚCallable)(ÚBaseImageProcessorr Ú DataCollatorÚ"DataCollatorForTokenClassificationÚDatasetÚEvalPredictionÚFeatureExtractionMixinr Ú PRMConfigÚ
|
|||
|
|
PRMTrainerÚ PartialStateÚPathÚ PeftModelÚPreTrainedModelÚPreTrainedTokenizerBaseÚProcessorMixinÚTrainerÚTrainerCallbackr ÚchainÚcompute_accuracyÚdisable_dropout_in_modelÚfeaturesÚgenerate_model_cardÚinspectÚis_peft_availableÚis_wandb_availableÚnnÚosÚ prepare_model_for_kbit_trainingÚtextwrapÚtorchÚwarningsr r r r r# r& r) )Ú*)Ú dataclassÚfield)ÚVersion)Ú nullcontext)ÚDataCollatorForSeq2SeqÚ DataCollatorForLanguageModelingTF)Úepilogue_fusionÚ max_autotuneÚ
|
|||
|
|
shape_paddingz
|
|||
|
|
trace.enabledztriton.cudagraphs)ÚdynamicÚ fullgraphÚoptionsc ó’ — t j | d| j d ¦ « dd¬¦ « }t j | d¦ « dd¬¦ « }g }t ||¦ « D ]‘\ }}| t j ¦ « }t j |d| d¦ « ¬¦ « d¦ « }t j
|
|||
|
|
|d¬¦ « }||z
|
|||
|
|
} | | ¦ « Œ’ t j |¦ « }| | j d | j d f¦ « }|S )Néÿÿÿÿé r )ÚchunksÚdim)r< Úindex)r< é )
|
|||
|
|
r) ÚchunkÚreshapeÚshapeÚzipÚtoÚfloat32ÚgatherÚ unsqueezeÚsqueezeÚ logsumexpÚappendÚconcat)
|
|||
|
|
Úlogitsr= Úchunked_logitsÚ
|
|||
|
|
chunked_indexÚall_per_token_logpsÚ chunk_logitsÚ chunk_indexÚselected_logitsÚlogsumexp_valuesÚper_token_logpss
|
|||
|
|
ú]/workspace/Fine-tuning/DS-LLM-TEMPLATE-FINETUNING/unsloth_compiled_cache/UnslothPRMTrainer.pyÚ chunked_selective_log_softmaxrU " s5 € õ ”[ § ¢ °°F´LÀÔ4DÑ!EÔ!EÐPQÐYZÐ[Ñ[Ô[€NÝ”[ § ¢ ¨rÑ!2Ô!2¸QÀaÐHÑHÔH€MØ Ðå%(¨ ¸ Ñ%GÔ%Gð 4ð 4Ñ!ˆ �kØ#—’¥u¤}Ñ5Ô5ˆ Ý œ, |¸2À{×G\ÒG\Ð]_ÑG`ÔG`ÐaÑaÔa×iÒiÐjlÑmÔmˆÝ œ?¨<¸rÐBÑBÔBÐØ)Ð,<Ñ<ˆØ×"Ò" ?Ñ3Ô3Ð3Ð3ØÝ œ,Ð':Ñ;Ô;ÐØ-×5Ò5°v´|ÀA´ÈÌ ÐUVÌÐ6XÑYÔYÐØ Ð ó c ó ‡ — e Zd ZU dZ edddi¬¦ « Zee ed< edddi¬¦ « Z ee
|
|||
|
|
ed < d,ˆ fd+„ Z ˆ xZ S )-ÚUnslothPRMConfiga:
|
|||
|
|
|
|||
|
|
Configuration class for the [`PRMTrainer`].
|
|||
|
|
|
|||
|
|
This class includes only the parameters that are specific to PRM training. For a full list of training arguments,
|
|||
|
|
please refer to the [`~transformers.TrainingArguments`] documentation. Note that default values in this class may
|
|||
|
|
differ from those in [`~transformers.TrainingArguments`].
|
|||
|
|
|
|||
|
|
Using [`~transformers.HfArgumentParser`] we can turn this class into
|
|||
|
|
[argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the
|
|||
|
|
command line.
|
|||
|
|
|
|||
|
|
Parameters:
|
|||
|
|
max_length (`int` or `None`, *optional*, defaults to `1024`):
|
|||
|
|
Maximum length of the sequences (prompt + completion) used for truncation.
|
|||
|
|
max_prompt_length (`int` or `None`, *optional*, defaults to `512`):
|
|||
|
|
Maximum length of the prompt used for truncation.
|
|||
|
|
max_completion_length (`int` or `None`, *optional*, defaults to `None`):
|
|||
|
|
Maximum length of the completion used for truncation. The completion is the concatenation of the steps.
|
|||
|
|
disable_dropout (`bool`, *optional*, defaults to `True`):
|
|||
|
|
Whether to disable dropout in the model.
|
|||
|
|
step_separator (`str`, *optional*, defaults to `"
|
|||
|
|
"`):
|
|||
|
|
Separator used to separate each step of the reasoning process.
|
|||
|
|
train_on_last_step_only (`bool`, *optional*, defaults to `False`):
|
|||
|
|
Whether to train only on the last step.
|
|||
|
|
dataset_num_proc (`int`, *optional*, defaults to `None`):
|
|||
|
|
Number of processes to use for processing the dataset.
|
|||
|
|
|
|||
|
|
NÚhelpzvLLM SamplingParams)ÚdefaultÚmetadataÚvllm_sampling_paramsr9 z8Chunk size to reduce memory usage. -1 is most efficient.Úunsloth_num_chunksFÚnor: é r éú ç-C ëâ6
|
|||
|
|
?ç{®Gáz„?çÍÌÌÌÌÌì?ç+‡ÙÎ÷ï?ç:Œ0âŽyE>ç ð?ç @Úlinearçš™™™™™¹?ÚpassiveÚwarningTÚstepsr> éô éO
|
|||
|
|
ÚO1ÚautoÚ ç Ú
|
|||
|
|
adamw_8bitÚlengthÚ
|
|||
|
|
every_saveÚlasté é é cŠ óN •— |dk rt d|› d�¦ « ‚|dk rt d|› d�¦ « ‚|€|#dk r
|
|||
|
|
|$dk rd}d }#|‡€!d
|
|||
|
|
d lm}‹ t |‹¦ « d z d ¦ « }‡ t ¦ « j d”i d
|
|||
|
|
|“d|“d|“d|“d|“d|“d|“d|“d| “d|
|
|||
|
|
“d| “d| “d|
|
|||
|
|
“d|“d|“d |“d |“d |“d |“d |“d!|“d"|“d#|“d$|“d%|“d&|“d'|“d(| “d)| “d*| “d+| “d,| “d-|!“d.|"“d/|#“d0|$“d1|%“d2|&“d3|'“d4|(“d5|)“d6|*“d7|+“d8|,“d9|-“d:|.“d;|/“d<|0“d=|1“d>|2“d?|3“d@|4“dA|5“dB|6“dC|7“dD|8“dE|9“dF|:“dG|;“dH|<“dI|=“dJ|>“dK|?“dL|@“dM|A“dN|B“dO|C“dP|D“dQ|E“dR|F“dS|G“dT|H“dU|I“dV|J“dW|K“dX|L“dY|M“dZ|N“d[|O“d\|P“d]|Q“d^|R“d_|S“d`|T“da|U“db|V“dc|W“dd|X“de|Y“df|Z“dg|[“dh|\“di|]“dj|^“dk|_“dl|`“dm|a“dn|b“do|c“dp|d“dq|e“dr|f“ds|g“dt|h“du|i“dv|j“dw|k“dx|l“dy|m“dz|n“d{|o“d||p“d}|q“d~|r“d|s“d€|t“d�|u“d‚|v“dƒ|w“d„|x“d…|y“d†|z“d‡|{“dˆ||“d‰|}“dŠ|~“d‹|“dŒ|€“d�|�“dŽ|‚“d�|ƒ“d�|„“d‘|…“d’|†“d“|‡“|Š¤Ž |ˆ| _ |‰| _ d S )•NgH¯¼šò×z>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!r> za` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!rl rm Ú unsloth_training_checkpointsr^ r )Ú cpu_countr_ Ú
|
|||
|
|
output_dirÚoverwrite_output_dirÚdo_trainÚdo_evalÚ
|
|||
|
|
do_predictÚ
|
|||
|
|
eval_strategyÚprediction_loss_onlyÚper_device_train_batch_sizeÚper_device_eval_batch_sizeÚper_gpu_train_batch_sizeÚper_gpu_eval_batch_sizeÚgradient_accumulation_stepsÚeval_accumulation_stepsÚ
|
|||
|
|
eval_delayÚtorch_empty_cache_stepsÚ
|
|||
|
|
learning_rateÚ weight_decayÚ
|
|||
|
|
adam_beta1Ú
|
|||
|
|
adam_beta2Ú adam_epsilonÚ
|
|||
|
|
max_grad_normÚnum_train_epochsÚ max_stepsÚlr_scheduler_typeÚ warmup_ratioÚ warmup_stepsÚ log_levelÚlog_level_replicaÚlog_on_each_nodeÚ logging_dirÚlogging_strategyÚlogging_first_stepÚ
|
|||
|
|
logging_stepsÚlogging_nan_inf_filterÚ
|
|||
|
|
save_strategyÚ
|
|||
|
|
save_stepsÚsave_total_limitÚsave_safetensorsÚsave_on_each_nodeÚsave_only_modelÚ'restore_callback_states_from_checkpointÚno_cudaÚuse_cpuÚuse_mps_deviceÚseedÚ data_seedÚ
|
|||
|
|
jit_mode_evalÚuse_ipexÚbf16Úfp16Úfp16_opt_levelÚhalf_precision_backendÚbf16_full_evalÚfp16_full_evalÚtf32Ú
|
|||
|
|
local_rankÚ ddp_backendÚ
|
|||
|
|
tpu_num_coresÚtpu_metrics_debugÚdebugÚdataloader_drop_lastÚ
|
|||
|
|
eval_stepsÚdataloader_num_workersÚdataloader_prefetch_factorÚ
|
|||
|
|
past_indexÚrun_nameÚ disable_tqdmÚremove_unused_columnsÚ label_namesÚload_best_model_at_endÚmetric_for_best_modelÚgreater_is_betterÚignore_data_skipÚfsdpÚfsdp_min_num_paramsÚ fsdp_configÚ"fsdp_transformer_layer_cls_to_wrapÚaccelerator_configÚ deepspeedÚlabel_smoothing_factorÚoptimÚ
|
|||
|
|
optim_argsÚ adafactorÚgroup_by_lengthÚlength_column_nameÚ report_toÚddp_find_unused_parametersÚddp_bucket_cap_mbÚddp_broadcast_buffersÚdataloader_pin_memoryÚ dataloader_persistent_workersÚskip_memory_metricsÚuse_legacy_prediction_loopÚ push_to_hubÚresume_from_checkpointÚ hub_model_idÚ hub_strategyÚ hub_tokenÚhub_private_repoÚhub_always_pushÚ hub_revisionÚgradient_checkpointingÚ gradient_checkpointing_kwargsÚinclude_inputs_for_metricsÚeval_do_concat_batchesÚ fp16_backendÚpush_to_hub_model_idÚpush_to_hub_organizationÚpush_to_hub_tokenÚ
|
|||
|
|
mp_parametersÚauto_find_batch_sizeÚfull_determinismÚ torchdynamoÚ ray_scopeÚ ddp_timeoutÚ
|
|||
|
|
torch_compileÚtorch_compile_backendÚtorch_compile_modeÚinclude_tokens_per_secondÚ include_num_input_tokens_seenÚneftune_noise_alphaÚoptim_target_modulesÚbatch_eval_metricsÚ
|
|||
|
|
eval_on_startÚuse_liger_kernelÚliger_kernel_configÚeval_use_gather_objectÚ average_tokens_across_devicesÚ
|
|||
|
|
max_lengthÚmax_prompt_lengthÚmax_completion_lengthÚdisable_dropoutÚstep_separatorÚtrain_on_last_step_onlyÚdataset_num_proc© ) ÚFloatingPointErrorÚ
|
|||
|
|
OverflowErrorÚmultiprocessingr| ÚminÚsuperÚ__init__r\ r] )�Úselfr} r~ r r€ r� r‚ rƒ r„ r… r† r‡ rˆ r‰ rŠ r‹ rŒ r� rŽ r� r� r‘ r’ r“ r” r• r– r— r˜ r™ rš r› rœ r� rž rŸ r r¡ r¢ r£ r¤ r¥ r¦ r§ r¨ r© rª r« r¬ r r® r¯ r° r± r² r³ r´ rµ r¶ r· r¸ r¹ rº r» r¼ r½ r¾ r¿ rÀ rÁ r rà rÄ rÅ rÆ rÇ rÈ rÉ rÊ rË rÌ rÍ rÎ rÏ rÐ rÑ rÒ rÓ rÔ rÕ rÖ r× rØ rÙ rÚ rÛ rÜ rÝ rÞ rß rà rá râ rã rä rå ræ rç rè ré rê rë rì rí rî rï rð rñ rò ró rô rõ rö r÷ rø rù rú rû rü rý rþ rÿ r r r r r\ r] Úkwargsr| Ú __class__s� €rT r
|
|||
|
|
zUnslothPRMConfig.__init__Z s ø€ ð\ ˜4Ò Ð Õ'9ð ;VÐ]jð ;Vð ;Vð ;Vñ (Wô (Wð "WØ ˜1Ò Ð ¥Mð 3FÐUbð 3Fð 3Fð 3Fñ %Gô %Gð GØ Ð -°7Ò":Ð":¸zÈSÒ?PÐ?PØ7ˆJØ ˆMØ Ð #Ø 1Ð 1Ð 1Ð 1Ð 1Ð 1Ý " 9 9¡;¤;¨q¡=°!Ñ 4Ô 4Ð à�‰ŒÔð G :ð G :ð G :Ø#˜ðG :à#7Ð#7ðG :ð �xðG :ð �gð G :ð
|
|||
|
|
$˜ð G :ð *˜Mð
|
|||
|
|
G :ð $8Ð#7ðG :ð +FÐ*EðG :ð *DÐ)CðG :ð (@Ð'?ðG :ð '>Ð&=ðG :ð +FÐ*EðG :ð '>Ð&=ðG :ð $˜ð G :ð '>Ð&=ð G :ð *˜Mð!G :ð" (˜<ð#G :ð$ $˜ð%G :ð& $˜ð'G :ð( (˜<ð)G :ð* *˜Mð+G :ð, 0Ð /ð-G :ð. "˜ ð/G :ð0 !2Ð 1ð1G :ð2 (˜<ð3G :ð4 (˜<ð5G :ð6 "˜ ð7G :ð8 !2Ð 1ð9G :ð: 0Ð /ð;G :ð< &˜+ð=G :ð> 0Ð /ð?G :ð@ "4Ð!3ðAG :ðB *˜MðCG :ðD &<Ð%;ðEG :ðF *˜MðGG :ðH $˜ðIG :ðJ 0Ð /ðKG :ðL 0Ð /ðMG :ðN !2Ð 1ðOG :ðP .˜oðQG :ðR 7^Ð6]ðSG :ðT �gðUG :ðV �gðWG :ðX ,˜^ðYG :ðZ �4ð[G :ð\ "˜ ð]G :ð^ *˜Mð_G :ð` �xðaG :ðb �4ðcG :ðd �4ðeG :ðf ,˜^ðgG :ðh &<Ð%;ðiG :ðj ,˜^ðkG :ðl ,˜^ðmG :ðn �4ðoG :ðp $˜ðqG :ðr &˜+ðsG :ðt *˜MðuG :ðv !2Ð 1ðwG :ðx �EðyG :ðz $8Ð#7ð{G :ð| $˜ð}G :ð~ &<Ð%;ðG :ð@ *DÐ)CðAG :ðB $˜ðCG :ðD �xðEG :ðF (˜<ðGG :ðH %:Ð$9ðIG :ðJ &˜+ðKG :ðL &<Ð%;ðMG :ðN %:Ð$9ðOG :ðP !2Ð 1ðQG :ðR 0Ð /ðSG :ðT �4ðUG :ðV #6Ð"5ðWG :ðX &˜+ðYG :ðZ 2TÐ1Sð[G :ð\ "4Ð!3ð]G :ð^ "˜ ð_G :ð` &<Ð%;ðaG :ðb �EðcG :ðd $˜ðeG :ðf "˜ ðgG :ðh .˜oðiG :ðj "4Ð!3ðkG :ðl "˜ ðmG :ðn *DÐ)CðoG :ðp !2Ð 1ðqG :ðr %:Ð$9ðsG :ðt %:Ð$9ðuG :ðv -JÐ,IðwG :ðx #6Ð"5ðyG :ðz *DÐ)Cð{G :ð| &˜+ð}G :ð~ &<Ð%;ðG :ð@ (˜<ðAG :ðB (˜<ðCG :ðD "˜ ðEG :ðF 0Ð /ðGG :ðH .˜oðIG :ðJ (˜<ðKG :ðL &<Ð%;ðMG :ðN -JÐ,IðOG :ðP *DÐ)CðQG :ðR &<Ð%;ðSG :ðT (˜<ðUG :ðV $8Ð#7ðWG :ðX (@Ð'?ðYG :ðZ !2Ð 1ð[G :ð\ *˜Mð]G :ð^ $8Ð#7ð_G :ð` 0Ð /ðaG :ðb &˜+ðcG :ðd "˜ ðeG :ðf &˜+ðgG :ðh *˜MðiG :ðj %:Ð$9ðkG :ðl "4Ð!3ðmG :ðn )BÐ(AðoG :ðp -JÐ,IðqG :ðr #6Ð"5ðsG :ðt $8Ð#7ðuG :ðv "4Ð!3ðwG :ðx *˜MðyG :ðz 0Ð /ð{G :ð| #6Ð"5ð}G :ð~ &<Ð%;ðG :ð@ -JÐ,IðAG :ðB $˜ðCG :ðD !2Ð 1ðEG :ðF %:Ð$9ðGG :ðH .˜oðIG :ðJ ,˜^ðKG :ðL '>Ð&=ðMG :ðN 0Ð /°&ðOG :ð G :ð G :ðP %9ˆÔ!Ø"4ˆÔ Ð Ð rV )‰NNFFFr^ Fr: r: NNr_ r_ r r` ra rb rc rd re rf rg r9 rh ri r rj rk TNrl Fr> Frl rm NTFFFFFFrn rn FFFFro rp FFNr9 NNFrq FNr Nr9 NNTNFNNFrq r NNNNrr rs NFFrt NNNNTFTFFNNru NNFNFNFTrp NNNrq TFNrv rw FNNFFNNFFFNFTrx ry NTrq FNNr9 )
|
|||
|
|
Ú__name__Ú
|
|||
|
|
__module__Ú __qualname__Ú__doc__r- r\ r r Ú__annotations__r] Úintr
|
|||
|
|
Ú
|
|||
|
|
__classcell__©r
|
|||
|
|
s @rT rX rX 3 sg ø€ € € € € € ð ð ð: +0¨%ØØÐ 1Ð2ð+ñ +ô +И( 3œ-ð ð ñ ð */¨ØØÐ VÐWð*ñ *ô *И #œ ð ð ñ ð Ø #ØØØØ Ø $Ø&'Ø%&Ø#'Ø"&Ø&'Ø"#ØØ"%Ø ØØØØ ØØ ØØ $ØØØ Ø %Ø ØØ"Ø "ØØ!&Ø ØØ Ø Ø !Ø Ø27ØØØ ØØØ ØØØØ Ø!'Ø Ø ØØØØ Ø !ØØ $ØØ!"Ø%)ØØØØ $ØØ!&Ø $Ø Ø ØØ ØØ-1Ø !ØØ!$Ø ØØØ Ø %ØØ%)Ø Ø $Ø $Ø(-Ø "Ø%*ØØ!%ØØ#ØØ Ø ØØ!&Ø(,Ø%*Ø!%Ø Ø #Ø#'Ø ØØ #Ø ØØØØ Ø $Ø !Ø$)Ø(-Ø "Ø #Ø "Ø Ø Ø "Ø!&Ø(,ØØ Ø $Ø ðà"'Ø Ø #Ø ðW`5ð `5ð `5ð `5ð `5ð `5ð `5ð `5ð `5ð `5rV rX c ó\ ‡ — e Zd ZdZddgZ ddeeeej f dee
|
|||
|
|
dee d ee d
|
|||
|
|
eee e
|
|||
|
|
ee f f d eeeeeef d eeg ef d
|
|||
|
|
eeege
|
|||
|
|
f deee deej j ej j j f deeej ej gej f dee
|
|||
|
|
fˆ fd„
|
|||
|
|
Z e d„ ¦ « Z ˆ fd„Z! ddee dee deeee df fd„Z"ˆ xZ#S )Ú_UnslothPRMTrainerrq ÚtrlÚprmN©NNÚmodelÚargsÚ
|
|||
|
|
data_collatorÚ
|
|||
|
|
train_datasetÚ eval_datasetÚprocessing_classÚ
|
|||
|
|
model_initÚcompute_metricsÚ callbacksÚ
|
|||
|
|
optimizersÚ preprocess_logits_for_metricsÚ peft_configc
|
|||
|
|
ó •— t ¦ « s| �t d¦ « ‚t ¦ « r¯| �t |t ¦ « s˜t |dd¦ « st |dd¦ « rtdt t
|
|||
|
|
j t ¦ « j ¦ « v }
|
|||
|
|
d|j
|
|||
|
|
i}|
|
|||
|
|
s |j �t j
|
|||
|
|
d¦ « n|
|
|||
|
|
r|j �
|
|||
|
|
|j |d< t |fi |¤Ž}|}|j rt |¦ « |€t }|€'|€t d¦ « ‚t# ||j ¬ ¦ « }d
|
|||
|
|
|j v�rzt) ¦ « ¦ « 5 ||j |j |j |j |j d œ}i |¥d di¥}| | j ||j |j d
|
|||
|
|
t; j t; j t; j d¦ « ¦ « t; j t; j d¦ « ¦ « dœ¦ « ¬¦ « }i |¥d di¥}|�‡| | j ||j |j dt; j t; j t; j d¦ « ¦ « t; j t; j d¦ « ¦ « dœ¦ « ¬¦ « }d d d ¦ « n # 1 swxY w Y tC ¦ « " ||||||||| |
|
|||
|
|
| ¬¦ « tG | j$ d¦ « r!| j$ % | j& ¦ « d S d S )NzvPEFT is not installed and you passed a `peft_config` in the trainer's kwargs, please install it to use the PEFT modelsÚis_loaded_in_8bitFÚ is_quantizedrã Úuse_gradient_checkpointingzÂYou passed `gradient_checkpointing_kwargs` in the trainer's kwargs, but your peft version does not support it. please update to the latest version of peft to use `gradient_checkpointing_kwargs`.z^A processing_class must be specified when using the default DataCollatorForTokenClassification)rý Ú input_ids)Ú tokenizerr rý rþ rÿ r Úis_evalzTokenizing train datasetÚint64)Úlabelsr+ )Ú fn_kwargsÚnum_procÚremove_columnsÚdescr TzTokenizing eval dataset) r r r r r r r! r" r# r$ r% Úadd_model_tags)'r# Ú
|
|||
|
|
ValueErrorÚ
|
|||
|
|
isinstancer ÚgetattrÚlistr" Ú signaturer' Ú
|
|||
|
|
parametersrâ rã r* Úwarnr r r r rý Ú column_namesr Úmain_process_firstr rþ rÿ r ÚmapÚ tokenize_rowr r ÚFeaturesÚSequenceÚValuer r
|
|||
|
|
Úhasattrr r4 Ú
|
|||
|
|
_tag_names)r r r r r r r r! r" r# r$ r% r& Ú_supports_gc_kwargsÚprepare_model_kwargsr0 Útrain_fn_kwargsÚeval_fn_kwargsr
|
|||
|
|
s €rT r
|
|||
|
|
z_UnslothPRMTrainer.__init__‚ sá ø€ õ( !Ñ"Ô"ð {Ð'>Ý ð Iñô ð
|
|||
|
|
õ Ñ
|
|||
|
|
Ô
|
|||
|
|
ð [Ð%<Ý ˜e¥YÑ/Ô/ð
|
|||
|
|
ݘ5Ð"5°uÑ=Ô=ð [ÅÈÐP^Ð`eÑAfÔAfð [Ø*IÍTÝ Ô)Õ*IÑJÔJÔUñNô Nð +Ð'ð -IÈ$ÔJeÐ+fÐ(à.ð s°4Ô3UÐ3aÝ œ
|
|||
|
|
ð rñô ð ð ð -ð s°Ô1SÐ1_ØPTÔPrÐ,Ð-LÑMå ;¸EÐ ZÐ ZÐEYÐ ZÐ Z�Eà �ð Ô ð ,Ý $ UÑ +Ô +Ð +à Ð "Ý .ˆOà Ð Ø Ð'Ý Øtñô ð õ ?Ð?OÐ\`Ô\kÐ lÑ lÔ lˆMà ˜mÔ 8Ð 8Ñ 8Ý ‘ ” ×2Ò2Ñ4Ô4ð &
|
|||
|
|
ð &
|
|||
|
|
à!1Ø&*Ô&9Ø"&¤/Ø)-Ô)?Ø-1Ô-GØ/3Ô/Kð
|
|||
|
|
ð � ð #B YÐ"A° ¸5Ð"AÐ"A�Ø -× 1Ò 1ØÔ%Ø -Ø !Ô 2Ø#0Ô#9Ø3Ý %Ô .å&.Ô&7½¼ÀwÑ8OÔ8OÑ&PÔ&PÝ)1Ô):½8¼>È'Ñ;RÔ;RÑ)SÔ)Sðð ñ ô ð
|
|||
|
|
!2ñ !ô !�
|
|||
|
|
ð "@ IÐ!?¨y¸$Ð!?Ð!?�Ø Ð+Ø#/×#3Ò#3Ø Ô)Ø"0Ø!%Ô!6Ø'3Ô'<Ø 6Ý!)Ô!2å*2Ô*;½H¼NÈ7Ñ<SÔ<SÑ*TÔ*TÝ-5Ô->½x¼~ÈgÑ?VÔ?VÑ-WÔ-Wð ð ñ"ô "ð
|
|||
|
|
$4ñ $ô $�Lð5&
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ñ &
|
|||
|
|
ô &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
øøøð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
ð &
|
|||
|
|
õP ‰Œ×ÒØØØ'Ø'Ø%Ø -Ø!Ø +Ø Ø!Ø*Gð ñ
|
|||
|
|
ô
|
|||
|
|
ð
|
|||
|
|
õ �4”:Ð /Ñ 0Ô 0ð 7Ø ŒJ× %Ò % d¤oÑ 6Ô 6Ð 6Ð 6Ð 6ð 7ð 7s ÅEJ Ê J#Ê&J#c ó¬ ‡‡
|
|||
|
|
— ‰| d d¬¦ « d }ˆfd„| d D ¦ « } |r<|s:dgt | d ¦ « d z
|
|||
|
|
z t | d d
|
|||
|
|
¦ « gz }
|
|||
|
|
nd „ | d D ¦ « }
|
|||
|
|
‰ |d¬¦ « Š
|
|||
|
|
ˆ
|
|||
|
|
fd „| D ¦ « } d
|
|||
|
|
„ t | |
|
|||
|
|
¦ « D ¦ « }
|
|||
|
|
t t | Ž ¦ « } t t |
|
|||
|
|
Ž ¦ « }
|
|||
|
|
‰j � ‰j g|z }|� || d… }|�| d|… } |
|
|||
|
|
d|… }
|
|||
|
|
|| z } dgt |¦ « z |
|
|||
|
|
z }
|
|||
|
|
|�| d|… } |
|
|||
|
|
d|… }
|
|||
|
|
| |
|
|||
|
|
dœS )a/
|
|||
|
|
Tokenize a row of the dataset.
|
|||
|
|
|
|||
|
|
Args:
|
|||
|
|
features (`dict[str, str]`):
|
|||
|
|
Row of the dataset, should contain the keys `"prompt"`, `"completions"`, and `"labels"`.
|
|||
|
|
tokenizer (`PreTrainedTokenizerBase`):
|
|||
|
|
Tokenizer used to process the data.
|
|||
|
|
step_separator (`str`):
|
|||
|
|
Separator between steps in the completion.
|
|||
|
|
max_length (`int` or `None`):
|
|||
|
|
Maximum length of the sequences (prompt + completion). If `None`, the sequences are not truncated.
|
|||
|
|
max_prompt_length (`int` or `None`):
|
|||
|
|
Maximum length of the prompt. If `None`, the prompt is not truncated.
|
|||
|
|
max_completion_length (`int` or `None`):
|
|||
|
|
Maximum length of the completion sequences. If `None`, the completion sequences are not truncated.
|
|||
|
|
train_on_last_step_only (`bool`):
|
|||
|
|
Whether to train only on the last step. If `True`, the labels are `-100` for all tokens except the last
|
|||
|
|
token of the completion.
|
|||
|
|
is_eval (`bool`):
|
|||
|
|
Whether the function is used to tokenize samples from a training or an evaluation dataset. Used only if
|
|||
|
|
`train_on_last_step_only` is set to `True`.
|
|||
|
|
|
|||
|
|
Returns:
|
|||
|
|
`dict[str, list[int]]`:
|
|||
|
|
Tokenized sequences with the keys `"input_ids"`, and `"labels".
|
|||
|
|
|
|||
|
|
Example:
|
|||
|
|
```python
|
|||
|
|
>>> from transformers import AutoTokenizer
|
|||
|
|
|
|||
|
|
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")
|
|||
|
|
>>> features = {
|
|||
|
|
... "prompt": "Which number is larger, 9.8 or 9.11?",
|
|||
|
|
... "completions": ["11 is greater than 8.", "Hence, 9.11 > 9.8."],
|
|||
|
|
... "labels": [True, False],
|
|||
|
|
... }
|
|||
|
|
>>> PRMTrainer.tokenize_row(
|
|||
|
|
... features, tokenizer, "\n", max_completion_length=None, train_on_last_step_only=False, is_eval=False
|
|||
|
|
... )
|
|||
|
|
{'input_ids': [23085, 1372, 374, 8131, 11, 220, 24, 13, 23, 476, 220, 24, 13, 16, 16, 30, 16, 16, 374, 7046, 1091, 220, 23, 13, 198, 39, 763, 11, 220, 24, 13, 16, 16, 861, 220, 24, 13, 23, 13, 198],
|
|||
|
|
'labels': [-100, -100, -100, -100, -100, -100, -100, -100, 1, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 0]}
|
|||
|
|
```
|
|||
|
|
ÚpromptF©Úadd_special_tokensr+ c ó6 •— g | ]} ‰|d ¬¦ « d ‘ŒS )FrK r+ r )Ú.0Ú
|
|||
|
|
completionr, s €rT ú
|
|||
|
|
<listcomp>z3_UnslothPRMTrainer.tokenize_row.<locals>.<listcomp>1 s: ø€ ð
|
|||
|
|
ð
|
|||
|
|
ð
|
|||
|
|
ØMWˆIˆI�j°UÐ ;Ñ ;Ô ;¸KÔ Hð
|
|||
|
|
ð
|
|||
|
|
ð
|
|||
|
|
rV Ú completionséœÿÿÿr/ r> r9 c ó, — g | ]}t |¦ « ‘ŒS r )r )rN Úlabels rT rP z3_UnslothPRMTrainer.tokenize_row.<locals>.<listcomp>7 s € ÐAÐAÐA U•c˜%‘j”jÐAÐAÐArV c ó •— g | ]}|‰z ‘ŒS r r )rN rO Ú
|
|||
|
|
separator_idss €rT rP z3_UnslothPRMTrainer.tokenize_row.<locals>.<listcomp>; s ø€ ÐXÐXÐX¸*˜:¨
|
|||
|
|
Ñ5ÐXÐXÐXrV c óH — g | ] \ }}d gt |¦ « dz
|
|||
|
|
z |gz ‘Œ S )rR r> )Úlen)rN rO rT s rT rP z3_UnslothPRMTrainer.tokenize_row.<locals>.<listcomp>> s6 € ÐqÐqÐqÑ?P¸zÈ5�4�&�C
|
|||
|
|
™OœO¨aÑ /Ñ0°E°7Ñ:ÐqÐqÐqrV N)r+ r/ )rX r ÚencoderB r8 r Ú bos_token_id)r r, r rý rþ rÿ r r- Ú
|
|||
|
|
prompt_idsÚcompletions_idsr/ Úcompletion_idsr+ rV s ` @rT r? z _UnslothPRMTrainer.tokenize_rowø sâ øø€ ðp �Y˜x¨Ô 1ÀeÐLÑLÔLÈ[ÔYˆ
|
|||
|
|
ð
|
|||
|
|
ð
|
|||
|
|
ð
|
|||
|
|
ð
|
|||
|
|
Ø[cÐdqÔ[rð
|
|||
|
|
ñ
|
|||
|
|
ô
|
|||
|
|
ˆð #ð B¨7ð BØ�V�s 8¨HÔ#5Ñ 6Ô 6¸Ñ :Ñ;½sÀ8ÈHÔCUÐVXÔCYÑ?ZÔ?ZÐ>[Ñ[ˆFˆFàAÐA¨h°xÔ.@ÐAÑAÔAˆFð "×(Ò(¨ ÈEÐ(ÑRÔRˆ
|
|||
|
|
ØXÐXÐXÐXÈÐXÑXÔXˆð rÐqÕTWÐXgÐioÑTpÔTpÐqÑqÔqˆõ �e _Ð 5Ñ6Ô6ˆÝ•e˜V�nÑ%Ô%ˆà Ô !Ð -Ø#Ô0Ð1°JÑ>ˆJð Ð (Ø#Ð%6Ð$6Ð$7Ð$7Ô8ˆJØ Ð ,Ø +Ð,BÐ-BÐ,BÔ CˆNØÐ 2Ð 2Ð 2Ô3ˆFà Ñ/ˆ Ø��#˜j™/œ/Ñ)¨FÑ2ˆà Ð !Ø! + : +Ô.ˆIؘK˜Z˜KÔ(ˆFà &°&Ð9Ð9Ð9rV c ó •— | j j € t | j j ¦ « j }n%| j j d¦ « d }| |¬¦ « t ¦ « ||¦ « d S )Nú/r9 )Ú
|
|||
|
|
model_name) r rÜ r r} ÚnameÚsplitÚcreate_model_cardr Ú_save_checkpoint)r r Útrialr` r
|
|||
|
|
s €rT rd z#_UnslothPRMTrainer._save_checkpointX sy ø€ Ø Œ9Ô !Ð )Ý ˜dœiÔ 2Ñ3Ô3Ô8ˆJˆJà œÔ/×5Ò5°cÑ:Ô:¸2Ô>ˆJØ × Ò ¨*Ð Ñ5Ô5Ð5Ý
|
|||
|
|
‰Œ× Ò ¨Ñ.Ô.Ð.Ð.Ð.rV r` Ú dataset_nameÚtagsc ó — | ¦ « sdS t | j j d¦ « r@t j | j j j ¦ « s| j j j }nd}|€t ¦ « }n(t |t ¦ « r|h}nt |¦ « }t | j j d¦ « r| d¦ « | | j
|
|||
|
|
¦ « t j d¦ « }t! ||| j ||t% ¦ « r t& j �t& j j ndd|d¬¦ « }| t j | j j d ¦ « ¦ « dS )
|
|||
|
|
aî
|
|||
|
|
Creates a draft of a model card using the information available to the `Trainer`.
|
|||
|
|
|
|||
|
|
Args:
|
|||
|
|
model_name (`str` or `None`, *optional*, defaults to `None`):
|