- Aug 17, 2016
- 540
- 2,364
I've released v2.075
Changed Full Batch logic "again", but if you do not use a big model you won't notice a difference.
Before v2.075 it worked like this:
Determine batch size by the values determined in the options menu.
Send the batch a maximum of 3 times.
If all 3 batch requests fail to give a usable response, split the batch and send the first and second half individually a maximum of 4 times each.
If a half fails completely do single line requests for in instead.
That worked fine for small models, because if the batch is only 40 lines, and half fails, 20 individual translation requests aren't a big deal.
But if you've actually sent 240 lines, and it split it to 120 halves, and one of them fails, having to do 120 individual requests to a premium provider really hurts.
So now it instead works like this:
Determine batch size by the values determined in the options menu.
Send the batch a maximum of 3 times.
If all 3 batch requests fail and half of the batch would still have more or the same lines as the Minimum Sent Lines value, it restarts the entire Full Batch loop with a changed Maximum Sent Lines value.
If half the batch line count would be more than the Minimum Sent Lines, it will set the Maximum Sent Lines value to the old batch line count x 0.5.
If half the batch line count would be less than the Minimum Sent Lines, but 3 quarters would still be more than the Mnimum Sent Lines, or 3 quarters would be exactly the Minimum Sent lines, it will set the Maximum Sent Lines to the old batch line count x 0.75.
If 3 quarters would still be below the Minimum Sent requests, it will set the Maximum Sent lines to the old batch line count x 0.5.
This repeats until half of the batch lines would be less than half of the Minimum Sent Lines, at which point it sends single line requests instead.
The moment it succeeded a batch or the single requests for the last attempt have finished, all values reset and it tries a normal full batch again.
That means in the case of the 240 batch failing with Minimum Sent Lines being 40.
After 3 failures It would become 120.
If it fails 3 times again it would become 60.
If it fails 3 times again it would become 45, because halving it would have put it below the Minimum Sent Lines.
If it fails 3 times again it would become 22.
If it fails 3 times again it would do single requests for the 22, because 11 would be less than half of Minimum Sent lines.
If we now assume that the entire requested translation was only 240, then the next batch would now be 218.
That doesn't sound great, but is almost absurd to actually happen that way with a large premium model, unless you grossly overestimate its abilities and you should be lowering those batch sizes significantly anyway.
It failing at 120 is already kinda weird, but if it would, it then doing 120 individual requests would suck so much that this could save a lot of time and tokens.
Changelog since last changelog post:
2.075
DSLR
Changed DSLR Full Batch logic to keep splitting batches during failure until the resulting batch would be below half of the Minimum Sent Lines.
Full Batch will now sent a full error report on the first retry instead of making another blank attempt. (Worse for small locally hosted models, but they don't charge for tokens.)
Adjusted Full Batch Prompts.
2.074
DSLR
\\i[num] commands will no longer be escaped even if they are at the start or end of a line (in case they are not actually used for pictures and needed for context).
Fixed some plugin texts being tagged green even though there is no Japanese to translate (if japanesesource is enabled).
Will no longer reject a response identical to the input if the input doesn't actually have something to translate.
Added escaping for \i[_icon] type constructs.
Fixed escaper still looking for the old placeholders.
Changed Full Batch logic "again", but if you do not use a big model you won't notice a difference.
Before v2.075 it worked like this:
Determine batch size by the values determined in the options menu.
Send the batch a maximum of 3 times.
If all 3 batch requests fail to give a usable response, split the batch and send the first and second half individually a maximum of 4 times each.
If a half fails completely do single line requests for in instead.
That worked fine for small models, because if the batch is only 40 lines, and half fails, 20 individual translation requests aren't a big deal.
But if you've actually sent 240 lines, and it split it to 120 halves, and one of them fails, having to do 120 individual requests to a premium provider really hurts.
So now it instead works like this:
Determine batch size by the values determined in the options menu.
Send the batch a maximum of 3 times.
If all 3 batch requests fail and half of the batch would still have more or the same lines as the Minimum Sent Lines value, it restarts the entire Full Batch loop with a changed Maximum Sent Lines value.
If half the batch line count would be more than the Minimum Sent Lines, it will set the Maximum Sent Lines value to the old batch line count x 0.5.
If half the batch line count would be less than the Minimum Sent Lines, but 3 quarters would still be more than the Mnimum Sent Lines, or 3 quarters would be exactly the Minimum Sent lines, it will set the Maximum Sent Lines to the old batch line count x 0.75.
If 3 quarters would still be below the Minimum Sent requests, it will set the Maximum Sent lines to the old batch line count x 0.5.
This repeats until half of the batch lines would be less than half of the Minimum Sent Lines, at which point it sends single line requests instead.
The moment it succeeded a batch or the single requests for the last attempt have finished, all values reset and it tries a normal full batch again.
That means in the case of the 240 batch failing with Minimum Sent Lines being 40.
After 3 failures It would become 120.
If it fails 3 times again it would become 60.
If it fails 3 times again it would become 45, because halving it would have put it below the Minimum Sent Lines.
If it fails 3 times again it would become 22.
If it fails 3 times again it would do single requests for the 22, because 11 would be less than half of Minimum Sent lines.
If we now assume that the entire requested translation was only 240, then the next batch would now be 218.
That doesn't sound great, but is almost absurd to actually happen that way with a large premium model, unless you grossly overestimate its abilities and you should be lowering those batch sizes significantly anyway.
It failing at 120 is already kinda weird, but if it would, it then doing 120 individual requests would suck so much that this could save a lot of time and tokens.
Changelog since last changelog post:
2.075
DSLR
Changed DSLR Full Batch logic to keep splitting batches during failure until the resulting batch would be below half of the Minimum Sent Lines.
Full Batch will now sent a full error report on the first retry instead of making another blank attempt. (Worse for small locally hosted models, but they don't charge for tokens.)
Adjusted Full Batch Prompts.
2.074
DSLR
\\i[num] commands will no longer be escaped even if they are at the start or end of a line (in case they are not actually used for pictures and needed for context).
Fixed some plugin texts being tagged green even though there is no Japanese to translate (if japanesesource is enabled).
Will no longer reject a response identical to the input if the input doesn't actually have something to translate.
Added escaping for \i[_icon] type constructs.
Fixed escaper still looking for the old placeholders.
