Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.870 Acc=0.24 ---23579.84 ms/batch
Step 00000100 L_out=1.144 Acc=0.43 ---45502.25 ms/batch
Step 00000150 L_out=0.773 Acc=0.35 ---67412.74 ms/batch
Step 00000200 L_out=0.815 Acc=0.19 ---89331.18 ms/batch
Step 00000250 L_out=0.452 Acc=0.12 ---111304.25 ms/batch
Step 00000300 L_out=0.473 Acc=0.30 ---133173.31 ms/batch
Step 00000350 L_out=0.940 Acc=0.12 ---155375.13 ms/batch
Step 00000400 L_out=1.011 Acc=0.38 ---177394.90 ms/batch
Step 00000450 L_out=0.603 Acc=0.14 ---199660.54 ms/batch
Step 00000500 L_out=0.359 Acc=0.44 ---221620.30 ms/batch
Step 00000550 L_out=0.199 Acc=0.64 ---243564.10 ms/batch
Step 00000600 L_out=0.398 Acc=0.70 ---265607.23 ms/batch
Step 00000650 L_out=0.453 Acc=0.79 ---287587.79 ms/batch
Round 2 | epoch=0, train costTime=295.0187966823578, | total_activation_sum=13886570
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.510 Acc=0.55 ---14514.59 ms/batch
Step 00000750 L_out=0.267 Acc=0.82 ---36228.71 ms/batch
Step 00000800 L_out=0.610 Acc=0.48 ---58158.37 ms/batch
Step 00000850 L_out=0.598 Acc=0.45 ---80449.40 ms/batch
Step 00000900 L_out=0.463 Acc=0.61 ---103734.10 ms/batch
Step 00000950 L_out=1.430 Acc=0.61 ---127152.96 ms/batch
Step 00001000 L_out=0.726 Acc=0.36 ---150508.61 ms/batch
Step 00001050 L_out=0.379 Acc=0.61 ---173926.32 ms/batch
Step 00001100 L_out=0.564 Acc=0.65 ---197493.12 ms/batch
Step 00001150 L_out=0.484 Acc=0.43 ---220690.81 ms/batch
Step 00001200 L_out=0.669 Acc=0.47 ---244016.05 ms/batch
Step 00001250 L_out=0.363 Acc=0.65 ---267280.54 ms/batch
Step 00001300 L_out=0.037 Acc=0.71 ---290811.68 ms/batch
Round 2 | epoch=1, train costTime=306.73845386505127, | total_activation_sum=11880923
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.709 Acc=0.65 --- 7395.60 ms/batch
Step 00001400 L_out=0.208 Acc=0.57 ---30807.30 ms/batch
Step 00001450 L_out=0.422 Acc=0.63 ---53997.97 ms/batch
Step 00001500 L_out=0.319 Acc=0.55 ---77373.37 ms/batch
Step 00001550 L_out=1.625 Acc=0.66 ---100638.89 ms/batch
Step 00001600 L_out=0.370 Acc=0.63 ---123889.43 ms/batch
Step 00001650 L_out=0.134 Acc=0.67 ---147383.40 ms/batch
Step 00001700 L_out=0.415 Acc=0.68 ---170856.29 ms/batch
Step 00001750 L_out=0.253 Acc=0.78 ---194192.00 ms/batch
Step 00001800 L_out=0.726 Acc=0.72 ---217476.98 ms/batch
Step 00001850 L_out=0.511 Acc=0.54 ---240774.24 ms/batch
Step 00001900 L_out=0.400 Acc=0.54 ---264296.74 ms/batch
Step 00001950 L_out=0.394 Acc=0.71 ---287735.30 ms/batch
Step 00002000 L_out=0.631 Acc=0.41 ---311306.50 ms/batch
Round 2 | epoch=2, train costTime=311.7684555053711, | total_activation_sum=12057916
Round 2 | ****EPOCH 3****
Step 00002050 L_out=1.076 Acc=0.62 ---22925.32 ms/batch
Step 00002100 L_out=0.543 Acc=0.58 ---46502.87 ms/batch
Step 00002150 L_out=0.375 Acc=0.64 ---69832.11 ms/batch
Step 00002200 L_out=0.333 Acc=0.63 ---93218.35 ms/batch
Step 00002250 L_out=0.699 Acc=0.59 ---116541.51 ms/batch
Step 00002300 L_out=0.419 Acc=0.73 ---139871.25 ms/batch
Step 00002350 L_out=0.327 Acc=0.64 ---162953.50 ms/batch
Step 00002400 L_out=0.517 Acc=0.63 ---186408.16 ms/batch
Step 00002450 L_out=0.336 Acc=0.67 ---209781.78 ms/batch
Step 00002500 L_out=0.641 Acc=0.54 ---233241.88 ms/batch
Step 00002550 L_out=0.161 Acc=0.90 ---256349.11 ms/batch
Step 00002600 L_out=0.500 Acc=0.59 ---279766.73 ms/batch
Step 00002650 L_out=0.360 Acc=0.69 ---303202.29 ms/batch
Round 2 | epoch=3, train costTime=311.5625159740448, | total_activation_sum=11936178
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.532 Acc=0.61 ---14885.85 ms/batch
Step 00002750 L_out=0.517 Acc=0.45 ---37988.64 ms/batch
Step 00002800 L_out=0.366 Acc=0.85 ---61247.23 ms/batch
Step 00002850 L_out=1.443 Acc=0.39 ---84659.18 ms/batch
Step 00002900 L_out=0.694 Acc=0.52 ---108001.26 ms/batch
Step 00002950 L_out=0.437 Acc=0.74 ---131292.57 ms/batch
Step 00003000 L_out=0.544 Acc=0.75 ---154609.14 ms/batch
Step 00003050 L_out=0.366 Acc=0.74 ---177783.62 ms/batch
Step 00003100 L_out=0.183 Acc=0.60 ---201260.72 ms/batch
Step 00003150 L_out=0.325 Acc=0.69 ---224367.59 ms/batch
Step 00003200 L_out=0.397 Acc=0.75 ---247812.69 ms/batch
Step 00003250 L_out=0.422 Acc=0.84 ---271431.24 ms/batch
Step 00003300 L_out=0.710 Acc=0.62 ---294511.00 ms/batch
Round 2 | epoch=4, train costTime=310.8360164165497, | total_activation_sum=12054971
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.162 Acc=0.72 --- 7039.88 ms/batch
Step 00003400 L_out=0.623 Acc=0.64 ---30327.42 ms/batch
Step 00003450 L_out=0.343 Acc=0.73 ---53751.56 ms/batch
Step 00003500 L_out=0.547 Acc=0.80 ---76909.15 ms/batch
Step 00003550 L_out=0.302 Acc=0.48 ---100091.74 ms/batch
Step 00003600 L_out=0.560 Acc=0.59 ---123447.35 ms/batch
Step 00003650 L_out=0.263 Acc=0.67 ---146767.79 ms/batch
Step 00003700 L_out=0.169 Acc=0.74 ---170029.17 ms/batch
Step 00003750 L_out=0.362 Acc=0.51 ---193455.57 ms/batch
Step 00003800 L_out=0.072 Acc=0.80 ---216771.74 ms/batch
Step 00003850 L_out=0.437 Acc=0.71 ---239898.42 ms/batch
Step 00003900 L_out=0.177 Acc=0.74 ---263140.25 ms/batch
Step 00003950 L_out=0.441 Acc=0.73 ---286211.26 ms/batch
Step 00004000 L_out=0.347 Acc=0.82 ---309567.81 ms/batch
Round 2 | epoch=5, train costTime=310.49546456336975, | total_activation_sum=11916628
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.143 Acc=0.77 ---22399.87 ms/batch
Step 00004100 L_out=0.307 Acc=0.84 ---45520.11 ms/batch
Step 00004150 L_out=0.154 Acc=0.77 ---68734.58 ms/batch
Step 00004200 L_out=0.453 Acc=0.79 ---91942.52 ms/batch
Step 00004250 L_out=0.290 Acc=0.83 ---115285.95 ms/batch
Step 00004300 L_out=0.021 Acc=0.74 ---138757.59 ms/batch
Step 00004350 L_out=0.262 Acc=0.71 ---161849.09 ms/batch
Step 00004400 L_out=0.224 Acc=0.78 ---185252.03 ms/batch
Step 00004450 L_out=0.353 Acc=0.61 ---208410.76 ms/batch
Step 00004500 L_out=0.243 Acc=0.91 ---231701.09 ms/batch
Step 00004550 L_out=0.312 Acc=0.65 ---255057.35 ms/batch
Step 00004600 L_out=0.115 Acc=0.83 ---278301.85 ms/batch
Step 00004650 L_out=0.422 Acc=0.70 ---301442.20 ms/batch
Round 2 | epoch=6, train costTime=310.25779938697815, | total_activation_sum=12106328
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.269 Acc=0.78 ---14318.42 ms/batch
Step 00004750 L_out=0.195 Acc=0.82 ---37652.01 ms/batch
Step 00004800 L_out=0.176 Acc=0.79 ---60886.69 ms/batch
Step 00004850 L_out=0.829 Acc=0.68 ---84281.40 ms/batch
Step 00004900 L_out=0.175 Acc=0.66 ---107497.88 ms/batch
Step 00004950 L_out=0.310 Acc=0.83 ---130692.96 ms/batch
Step 00005000 L_out=0.529 Acc=0.80 ---154086.61 ms/batch
Step 00005050 L_out=0.201 Acc=0.63 ---177356.05 ms/batch
Step 00005100 L_out=0.223 Acc=0.70 ---200481.07 ms/batch
Step 00005150 L_out=0.171 Acc=0.83 ---223731.10 ms/batch
Step 00005200 L_out=2.378 Acc=0.73 ---247052.14 ms/batch
Step 00005250 L_out=0.211 Acc=0.78 ---270391.54 ms/batch
Step 00005300 L_out=0.177 Acc=0.85 ---293627.32 ms/batch
Round 2 | epoch=7, train costTime=310.4741222858429, | total_activation_sum=12001347
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.620 Acc=0.79 --- 6454.41 ms/batch
Step 00005400 L_out=0.189 Acc=0.85 ---29781.54 ms/batch
Step 00005450 L_out=0.978 Acc=0.69 ---53037.21 ms/batch
Step 00005500 L_out=0.457 Acc=0.76 ---76256.69 ms/batch
Step 00005550 L_out=0.672 Acc=0.76 ---99210.70 ms/batch
Step 00005600 L_out=0.322 Acc=0.82 ---122370.64 ms/batch
Step 00005650 L_out=0.299 Acc=0.79 ---145547.85 ms/batch
Step 00005700 L_out=0.164 Acc=0.88 ---168957.82 ms/batch
Step 00005750 L_out=0.115 Acc=0.80 ---192326.89 ms/batch
Step 00005800 L_out=0.303 Acc=0.81 ---215477.05 ms/batch
Step 00005850 L_out=0.610 Acc=0.74 ---238737.48 ms/batch
Step 00005900 L_out=0.277 Acc=0.82 ---261985.63 ms/batch
Step 00005950 L_out=0.171 Acc=0.79 ---285160.19 ms/batch
Step 00006000 L_out=0.667 Acc=0.78 ---308574.08 ms/batch
Round 2 | epoch=8, train costTime=309.9735984802246, | total_activation_sum=11945312
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.289 Acc=0.75 ---21952.27 ms/batch
Step 00006100 L_out=0.036 Acc=0.63 ---45179.10 ms/batch
Step 00006150 L_out=0.063 Acc=0.82 ---68520.89 ms/batch
Step 00006200 L_out=0.269 Acc=0.85 ---91874.24 ms/batch
Step 00006250 L_out=0.025 Acc=0.85 ---115174.61 ms/batch
Step 00006300 L_out=0.105 Acc=0.87 ---138349.86 ms/batch
Step 00006350 L_out=0.121 Acc=0.84 ---161649.19 ms/batch
Step 00006400 L_out=0.259 Acc=0.85 ---184767.90 ms/batch
Step 00006450 L_out=0.275 Acc=0.85 ---208076.27 ms/batch
Step 00006500 L_out=0.343 Acc=0.88 ---231271.59 ms/batch
Step 00006550 L_out=0.236 Acc=0.89 ---254456.26 ms/batch
Step 00006600 L_out=0.278 Acc=0.80 ---277640.10 ms/batch
Step 00006650 L_out=0.216 Acc=0.82 ---300785.86 ms/batch
Round 2 | epoch=9, train costTime=309.89699482917786, | total_activation_sum=11976778
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.105 Acc=0.80 ---14006.06 ms/batch
Step 00006750 L_out=0.093 Acc=0.88 ---37374.21 ms/batch
Step 00006800 L_out=0.206 Acc=0.74 ---60595.87 ms/batch
Step 00006850 L_out=0.325 Acc=0.76 ---83055.98 ms/batch
Step 00006900 L_out=0.159 Acc=0.91 ---105466.08 ms/batch
Step 00006950 L_out=0.267 Acc=0.82 ---128703.99 ms/batch
Step 00007000 L_out=0.192 Acc=0.89 ---151810.68 ms/batch
Step 00007050 L_out=0.400 Acc=0.62 ---175081.11 ms/batch
Step 00007100 L_out=0.146 Acc=0.82 ---198258.47 ms/batch
Step 00007150 L_out=0.375 Acc=0.83 ---221102.40 ms/batch
Step 00007200 L_out=0.090 Acc=0.84 ---244357.60 ms/batch
Step 00007250 L_out=0.515 Acc=0.80 ---267475.99 ms/batch
Step 00007300 L_out=0.363 Acc=0.68 ---290489.51 ms/batch
Round 2 | epoch=10, train costTime=307.7333357334137, | total_activation_sum=12129568
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.127 Acc=0.88 --- 6068.05 ms/batch
Step 00007400 L_out=0.301 Acc=0.83 ---29306.33 ms/batch
Step 00007450 L_out=0.267 Acc=0.82 ---52601.21 ms/batch
Step 00007500 L_out=1.020 Acc=0.82 ---75813.65 ms/batch
Step 00007550 L_out=0.236 Acc=0.72 ---98756.88 ms/batch
Step 00007600 L_out=0.101 Acc=0.53 ---122029.71 ms/batch
Step 00007650 L_out=0.161 Acc=0.72 ---145157.99 ms/batch
Step 00007700 L_out=0.304 Acc=0.80 ---168120.13 ms/batch
Step 00007750 L_out=0.105 Acc=0.83 ---191511.91 ms/batch
Step 00007800 L_out=0.126 Acc=0.87 ---214580.51 ms/batch
Step 00007850 L_out=0.069 Acc=0.76 ---237878.25 ms/batch
Step 00007900 L_out=0.265 Acc=0.77 ---260955.42 ms/batch
Step 00007950 L_out=0.132 Acc=0.87 ---284219.39 ms/batch
Step 00008000 L_out=0.286 Acc=0.83 ---307424.16 ms/batch
Round 2 | epoch=11, train costTime=309.25176215171814, | total_activation_sum=12090241
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.223 Acc=0.84 ---21259.00 ms/batch
Step 00008100 L_out=0.139 Acc=0.89 ---44319.49 ms/batch
Step 00008150 L_out=0.267 Acc=0.73 ---67565.71 ms/batch
Step 00008200 L_out=0.162 Acc=0.89 ---90648.23 ms/batch
Step 00008250 L_out=0.479 Acc=0.83 ---113677.38 ms/batch
Step 00008300 L_out=0.291 Acc=0.76 ---137209.13 ms/batch
Step 00008350 L_out=0.282 Acc=0.81 ---160506.40 ms/batch
Step 00008400 L_out=0.191 Acc=0.81 ---183807.87 ms/batch
Step 00008450 L_out=0.164 Acc=0.81 ---206923.70 ms/batch
Step 00008500 L_out=0.051 Acc=0.90 ---230007.97 ms/batch
Step 00008550 L_out=0.274 Acc=0.72 ---253410.56 ms/batch
Step 00008600 L_out=0.365 Acc=0.73 ---276829.86 ms/batch
Step 00008650 L_out=0.216 Acc=0.67 ---299943.47 ms/batch
Round 2 | epoch=12, train costTime=309.5406837463379, | total_activation_sum=11997332
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.045 Acc=0.90 ---13461.19 ms/batch
Step 00008750 L_out=0.509 Acc=0.82 ---36554.03 ms/batch
Step 00008800 L_out=0.262 Acc=0.83 ---59712.71 ms/batch
Step 00008850 L_out=0.099 Acc=0.88 ---83003.10 ms/batch
Step 00008900 L_out=0.156 Acc=0.86 ---106062.71 ms/batch
Step 00008950 L_out=0.314 Acc=0.88 ---129432.91 ms/batch
Step 00009000 L_out=0.314 Acc=0.86 ---152586.42 ms/batch
Step 00009050 L_out=0.099 Acc=0.92 ---175651.23 ms/batch
Step 00009100 L_out=0.107 Acc=0.84 ---198682.39 ms/batch
Step 00009150 L_out=0.284 Acc=0.92 ---222073.37 ms/batch
Step 00009200 L_out=0.163 Acc=0.86 ---245048.94 ms/batch
Step 00009250 L_out=0.321 Acc=0.79 ---268202.64 ms/batch
Step 00009300 L_out=0.157 Acc=0.88 ---291366.54 ms/batch
Round 2 | epoch=13, train costTime=308.9097464084625, | total_activation_sum=12054659
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.464 Acc=0.77 --- 5589.26 ms/batch
Step 00009400 L_out=0.481 Acc=0.69 ---29077.63 ms/batch
Step 00009450 L_out=0.193 Acc=0.88 ---52329.88 ms/batch
Step 00009500 L_out=0.465 Acc=0.82 ---75533.15 ms/batch
Step 00009550 L_out=0.242 Acc=0.78 ---99042.07 ms/batch
Step 00009600 L_out=0.137 Acc=0.87 ---122072.41 ms/batch
Step 00009650 L_out=0.090 Acc=0.90 ---145367.30 ms/batch
Step 00009700 L_out=0.278 Acc=0.81 ---168499.02 ms/batch
Step 00009750 L_out=0.388 Acc=0.80 ---191695.76 ms/batch
Step 00009800 L_out=0.108 Acc=0.91 ---214984.83 ms/batch
Step 00009850 L_out=0.371 Acc=0.72 ---238265.29 ms/batch
Step 00009900 L_out=0.524 Acc=0.70 ---261611.42 ms/batch
Step 00009950 L_out=1.120 Acc=0.69 ---284670.49 ms/batch
Step 00010000 L_out=0.259 Acc=0.86 ---307851.53 ms/batch
Round 2 | epoch=14, train costTime=310.1944921016693, | total_activation_sum=12075680
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.242 Acc=0.85 ---21012.42 ms/batch
Step 00010100 L_out=0.209 Acc=0.83 ---44153.93 ms/batch
Step 00010150 L_out=0.086 Acc=0.90 ---67434.24 ms/batch
Step 00010200 L_out=0.196 Acc=0.82 ---90715.25 ms/batch
Step 00010250 L_out=0.101 Acc=0.83 ---113957.86 ms/batch
Step 00010300 L_out=0.110 Acc=0.84 ---137141.75 ms/batch
Step 00010350 L_out=0.183 Acc=0.87 ---160408.96 ms/batch
Step 00010400 L_out=0.132 Acc=0.85 ---183653.86 ms/batch
Step 00010450 L_out=0.199 Acc=0.77 ---206772.88 ms/batch
Step 00010500 L_out=0.185 Acc=0.79 ---230148.14 ms/batch
Step 00010550 L_out=0.242 Acc=0.91 ---253488.22 ms/batch
Step 00010600 L_out=0.168 Acc=0.86 ---276716.13 ms/batch
Step 00010650 L_out=0.119 Acc=0.76 ---300253.72 ms/batch
Round 2 | epoch=15, train costTime=310.41814947128296, | total_activation_sum=12086063
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.243 Acc=0.89 ---12801.73 ms/batch
Step 00010750 L_out=0.101 Acc=0.89 ---35805.82 ms/batch
Step 00010800 L_out=0.410 Acc=0.77 ---59107.42 ms/batch
Step 00010850 L_out=0.255 Acc=0.89 ---82329.23 ms/batch
Step 00010900 L_out=0.145 Acc=0.92 ---105716.60 ms/batch
Step 00010950 L_out=0.864 Acc=0.79 ---129046.09 ms/batch
Step 00011000 L_out=0.127 Acc=0.81 ---152126.50 ms/batch
Step 00011050 L_out=0.103 Acc=0.86 ---175391.63 ms/batch
Step 00011100 L_out=0.106 Acc=0.86 ---198608.21 ms/batch
Step 00011150 L_out=0.179 Acc=0.90 ---221914.35 ms/batch
Step 00011200 L_out=0.126 Acc=0.88 ---245263.59 ms/batch
Step 00011250 L_out=0.178 Acc=0.77 ---268409.09 ms/batch
Step 00011300 L_out=0.097 Acc=0.89 ---291644.57 ms/batch
Round 2 | epoch=16, train costTime=309.65450620651245, | total_activation_sum=12021527
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.097 Acc=0.71 --- 5126.05 ms/batch
Step 00011400 L_out=0.057 Acc=0.84 ---28299.81 ms/batch
Step 00011450 L_out=0.135 Acc=0.86 ---51470.85 ms/batch
Step 00011500 L_out=0.111 Acc=0.85 ---74514.12 ms/batch
Step 00011550 L_out=0.321 Acc=0.85 ---97619.61 ms/batch
Step 00011600 L_out=0.199 Acc=0.86 ---120741.22 ms/batch
Step 00011650 L_out=0.018 Acc=0.84 ---143842.84 ms/batch
Step 00011700 L_out=0.097 Acc=0.87 ---167096.00 ms/batch
Step 00011750 L_out=0.066 Acc=0.80 ---190334.20 ms/batch
Step 00011800 L_out=0.164 Acc=0.88 ---213505.10 ms/batch
Step 00011850 L_out=0.207 Acc=0.85 ---236859.08 ms/batch
Step 00011900 L_out=0.111 Acc=0.92 ---259899.50 ms/batch
Step 00011950 L_out=0.222 Acc=0.91 ---282992.29 ms/batch
Step 00012000 L_out=0.175 Acc=0.90 ---306130.10 ms/batch
Round 2 | epoch=17, train costTime=308.9709186553955, | total_activation_sum=12023133
Round 2 | epoch=18, current m_iou=0.6329454682513386
Round 2 | Best m_IoU is: 0.633, OA is: 0.899 | val costTime=143.38418531417847
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.228 Acc=0.90 ---20410.56 ms/batch
Step 00012100 L_out=0.092 Acc=0.89 ---43710.31 ms/batch
Step 00012150 L_out=0.126 Acc=0.89 ---66963.18 ms/batch
Step 00012200 L_out=0.363 Acc=0.71 ---90109.73 ms/batch
Step 00012250 L_out=0.303 Acc=0.83 ---113165.19 ms/batch
Step 00012300 L_out=0.085 Acc=0.88 ---136569.73 ms/batch
Step 00012350 L_out=0.112 Acc=0.88 ---159727.72 ms/batch
Step 00012400 L_out=0.209 Acc=0.81 ---183088.33 ms/batch
Step 00012450 L_out=0.227 Acc=0.79 ---206255.82 ms/batch
Step 00012500 L_out=0.166 Acc=0.93 ---229427.41 ms/batch
Step 00012550 L_out=0.208 Acc=0.79 ---252714.21 ms/batch
Step 00012600 L_out=0.069 Acc=0.86 ---275842.97 ms/batch
Step 00012650 L_out=0.066 Acc=0.79 ---299088.90 ms/batch
Round 2 | epoch=18, train costTime=309.9047360420227, | total_activation_sum=12020844
Round 2 | epoch=19, current m_iou=0.6437358634104069
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=142.80502462387085
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.212 Acc=0.89 ---12635.99 ms/batch
Step 00012750 L_out=0.071 Acc=0.92 ---35953.56 ms/batch
Step 00012800 L_out=0.085 Acc=0.92 ---58557.16 ms/batch
Step 00012850 L_out=0.068 Acc=0.88 ---80767.71 ms/batch
Step 00012900 L_out=0.051 Acc=0.89 ---103397.39 ms/batch
Step 00012950 L_out=0.111 Acc=0.92 ---126715.38 ms/batch
Step 00013000 L_out=0.097 Acc=0.75 ---149795.49 ms/batch
Step 00013050 L_out=0.078 Acc=0.92 ---173254.82 ms/batch
Step 00013100 L_out=0.228 Acc=0.89 ---196511.76 ms/batch
Step 00013150 L_out=0.184 Acc=0.83 ---219514.51 ms/batch
Step 00013200 L_out=0.043 Acc=0.91 ---242699.33 ms/batch
Step 00013250 L_out=0.108 Acc=0.90 ---266166.71 ms/batch
Step 00013300 L_out=0.081 Acc=0.90 ---289369.97 ms/batch
Round 2 | epoch=19, train costTime=307.83594584465027, | total_activation_sum=12119365
Round 2 | epoch=20, current m_iou=0.6425519412658514
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=155.93478512763977
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.171 Acc=0.86 --- 4618.06 ms/batch
Step 00013400 L_out=0.402 Acc=0.91 ---27917.01 ms/batch
Step 00013450 L_out=0.092 Acc=0.92 ---51044.06 ms/batch
Step 00013500 L_out=0.148 Acc=0.86 ---74279.53 ms/batch
Step 00013550 L_out=0.175 Acc=0.79 ---97334.39 ms/batch
Step 00013600 L_out=0.109 Acc=0.85 ---120516.07 ms/batch
Step 00013650 L_out=0.031 Acc=0.84 ---144041.67 ms/batch
Step 00013700 L_out=0.127 Acc=0.86 ---167289.75 ms/batch
Step 00013750 L_out=0.055 Acc=0.86 ---190587.58 ms/batch
Step 00013800 L_out=0.446 Acc=0.82 ---213589.41 ms/batch
Step 00013850 L_out=0.482 Acc=0.72 ---236738.31 ms/batch
Step 00013900 L_out=0.048 Acc=0.93 ---260078.44 ms/batch
Step 00013950 L_out=0.089 Acc=0.91 ---283457.07 ms/batch
Step 00014000 L_out=0.072 Acc=0.93 ---306922.29 ms/batch
Round 2 | epoch=20, train costTime=310.16157817840576, | total_activation_sum=12021871
Round 2 | epoch=21, current m_iou=0.6344570879917708
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=146.26573705673218
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.212 Acc=0.80 ---20309.86 ms/batch
Step 00014100 L_out=0.090 Acc=0.90 ---43948.92 ms/batch
Step 00014150 L_out=0.119 Acc=0.95 ---67369.59 ms/batch
Step 00014200 L_out=0.180 Acc=0.83 ---90961.98 ms/batch
Step 00014250 L_out=0.013 Acc=0.95 ---114653.84 ms/batch
Step 00014300 L_out=0.017 Acc=0.91 ---138342.54 ms/batch
Step 00014350 L_out=0.057 Acc=0.90 ---161773.87 ms/batch
Step 00014400 L_out=0.060 Acc=0.94 ---185570.45 ms/batch
Step 00014450 L_out=0.085 Acc=0.94 ---209049.88 ms/batch
Step 00014500 L_out=0.120 Acc=0.89 ---232598.03 ms/batch
Step 00014550 L_out=0.050 Acc=0.91 ---256141.00 ms/batch
Step 00014600 L_out=0.071 Acc=0.90 ---279772.48 ms/batch
Step 00014650 L_out=0.225 Acc=0.85 ---303110.07 ms/batch
Round 2 | epoch=21, train costTime=314.45442748069763, | total_activation_sum=11978848
Round 2 | epoch=22, current m_iou=0.6375922182354452
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=128.7145857810974
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.103 Acc=0.83 ---12242.99 ms/batch
Step 00014750 L_out=0.109 Acc=0.90 ---35318.05 ms/batch
Step 00014800 L_out=0.118 Acc=0.85 ---58748.02 ms/batch
Step 00014850 L_out=2.230 Acc=0.55 ---99322.58 ms/batch
Step 00014900 L_out=0.115 Acc=0.91 ---122946.44 ms/batch
Step 00014950 L_out=0.090 Acc=0.80 ---146352.95 ms/batch
Step 00015000 L_out=0.036 Acc=0.89 ---168822.54 ms/batch
Step 00015050 L_out=0.136 Acc=0.86 ---192034.19 ms/batch
Step 00015100 L_out=0.060 Acc=0.85 ---215233.15 ms/batch
Step 00015150 L_out=0.172 Acc=0.91 ---238445.30 ms/batch
Step 00015200 L_out=0.078 Acc=0.92 ---261818.41 ms/batch
Step 00015250 L_out=0.059 Acc=0.89 ---308759.22 ms/batch
Step 00015300 L_out=0.156 Acc=0.94 ---331294.03 ms/batch
Round 2 | epoch=22, train costTime=350.36244344711304, | total_activation_sum=12197892
Round 2 | epoch=23, current m_iou=0.6392616541249552
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=140.5578305721283
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.268 Acc=0.81 --- 3982.27 ms/batch
Step 00015400 L_out=0.257 Acc=0.77 ---26538.74 ms/batch
Step 00015450 L_out=0.125 Acc=0.91 ---49754.66 ms/batch
Step 00015500 L_out=0.110 Acc=0.80 ---72723.55 ms/batch
Step 00015550 L_out=0.137 Acc=0.91 ---95772.10 ms/batch
Step 00015600 L_out=0.052 Acc=0.94 ---118769.90 ms/batch
Step 00015650 L_out=0.083 Acc=0.84 ---141750.77 ms/batch
Step 00015700 L_out=0.093 Acc=0.93 ---164856.47 ms/batch
Step 00015750 L_out=0.098 Acc=0.89 ---258366.67 ms/batch
Step 00015800 L_out=0.164 Acc=0.86 ---281061.88 ms/batch
Step 00015850 L_out=0.028 Acc=0.88 ---304013.21 ms/batch
Step 00015900 L_out=0.085 Acc=0.90 ---327256.17 ms/batch
Step 00015950 L_out=0.034 Acc=0.80 ---350456.73 ms/batch
Step 00016000 L_out=0.352 Acc=0.81 ---373569.29 ms/batch
Round 2 | epoch=23, train costTime=377.2691104412079, | total_activation_sum=12140591
Round 2 | epoch=24, current m_iou=0.6365598658998481
Round 2 | Best m_IoU is: 0.644, OA is: 0.907 | val costTime=155.00011372566223
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.088 Acc=0.82 ---19366.97 ms/batch
Step 00016100 L_out=0.172 Acc=0.85 ---41871.39 ms/batch
Step 00016150 L_out=0.035 Acc=0.92 ---64822.36 ms/batch
Step 00016200 L_out=0.178 Acc=0.87 ---87834.08 ms/batch
Step 00016250 L_out=0.044 Acc=0.84 ---110631.37 ms/batch
Step 00016300 L_out=0.070 Acc=0.90 ---133316.69 ms/batch
Step 00016350 L_out=0.146 Acc=0.86 ---156246.51 ms/batch
Step 00016400 L_out=0.261 Acc=0.84 ---179015.75 ms/batch
Step 00016450 L_out=0.184 Acc=0.94 ---201788.20 ms/batch
Step 00016500 L_out=0.124 Acc=0.90 ---224438.20 ms/batch
Step 00016550 L_out=0.131 Acc=0.91 ---247337.31 ms/batch
Step 00016600 L_out=0.069 Acc=0.89 ---429743.22 ms/batch
Step 00016650 L_out=0.016 Acc=0.92 ---454833.92 ms/batch
Round 2 | epoch=24, train costTime=466.8139944076538, | total_activation_sum=12005490
Round 2 | epoch=25, current m_iou=0.6437748992138166
Round 2 | Best m_IoU is: 0.644, OA is: 0.906 | val costTime=118.05745124816895
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.115 Acc=0.94 ---11149.62 ms/batch
Step 00016750 L_out=0.106 Acc=0.83 ---33570.27 ms/batch
Step 00016800 L_out=0.103 Acc=0.86 ---56257.43 ms/batch
Step 00016850 L_out=0.122 Acc=0.85 ---78928.83 ms/batch
Step 00016900 L_out=0.076 Acc=0.89 ---101789.62 ms/batch
Step 00016950 L_out=0.015 Acc=0.95 ---124615.89 ms/batch
Step 00017000 L_out=0.136 Acc=0.93 ---147309.91 ms/batch
Step 00017050 L_out=0.204 Acc=0.86 ---170232.84 ms/batch
Step 00017100 L_out=0.070 Acc=0.81 ---193060.41 ms/batch
Step 00017150 L_out=0.167 Acc=0.83 ---216036.76 ms/batch
Step 00017200 L_out=0.037 Acc=0.89 ---238891.84 ms/batch
Step 00017250 L_out=0.027 Acc=0.90 ---263253.19 ms/batch
Step 00017300 L_out=0.250 Acc=0.87 ---287024.54 ms/batch
Round 2 | epoch=25, train costTime=306.93295073509216, | total_activation_sum=12005933
Round 2 | epoch=26, current m_iou=0.643686778469475
Round 2 | Best m_IoU is: 0.644, OA is: 0.906 | val costTime=124.57902383804321
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.223 Acc=0.87 --- 4693.06 ms/batch
Step 00017400 L_out=0.060 Acc=0.87 ---32170.25 ms/batch
Step 00017450 L_out=0.129 Acc=0.88 ---57554.96 ms/batch
Step 00017500 L_out=0.052 Acc=0.94 ---84928.79 ms/batch
Step 00017550 L_out=0.277 Acc=0.89 ---109217.41 ms/batch
Step 00017600 L_out=0.189 Acc=0.89 ---132907.83 ms/batch
Step 00017650 L_out=0.056 Acc=0.89 ---156795.24 ms/batch
Step 00017700 L_out=0.089 Acc=0.82 ---180686.01 ms/batch
Step 00017750 L_out=0.086 Acc=0.92 ---204351.68 ms/batch
Step 00017800 L_out=0.058 Acc=0.94 ---228152.99 ms/batch
Step 00017850 L_out=0.097 Acc=0.88 ---252084.05 ms/batch
Step 00017900 L_out=0.062 Acc=0.86 ---275975.78 ms/batch
Step 00017950 L_out=0.110 Acc=0.92 ---299721.34 ms/batch
Step 00018000 L_out=0.088 Acc=0.88 ---323389.71 ms/batch
Round 2 | epoch=26, train costTime=327.7192642688751, | total_activation_sum=11851715
Round 2 | epoch=27, current m_iou=0.6420401103993323
Round 2 | Best m_IoU is: 0.644, OA is: 0.906 | val costTime=117.70609283447266
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.083 Acc=0.90 ---19387.50 ms/batch
Step 00018100 L_out=0.155 Acc=0.87 ---43250.92 ms/batch
Step 00018150 L_out=0.027 Acc=0.90 ---66963.60 ms/batch
Step 00018200 L_out=0.212 Acc=0.88 ---90214.73 ms/batch
Step 00018250 L_out=0.052 Acc=0.95 ---113122.67 ms/batch
Step 00018300 L_out=0.003 Acc=0.74 ---139278.92 ms/batch
Step 00018350 L_out=0.057 Acc=0.90 ---163477.34 ms/batch
Step 00018400 L_out=0.147 Acc=0.92 ---189502.80 ms/batch
Step 00018450 L_out=0.038 Acc=0.86 ---216729.95 ms/batch
Step 00018500 L_out=0.055 Acc=0.89 ---239980.47 ms/batch
Step 00018550 L_out=0.089 Acc=0.92 ---262526.74 ms/batch
Step 00018600 L_out=0.046 Acc=0.96 ---289607.84 ms/batch
Step 00018650 L_out=0.125 Acc=0.89 ---314264.21 ms/batch
Round 2 | epoch=27, train costTime=327.75939321517944, | total_activation_sum=12037357
Round 2 | epoch=28, current m_iou=0.6415900990656445
Round 2 | Best m_IoU is: 0.644, OA is: 0.906 | val costTime=146.156991481781
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.029 Acc=0.62 ---11451.48 ms/batch
Step 00018750 L_out=0.103 Acc=0.92 ---34384.76 ms/batch
Step 00018800 L_out=0.123 Acc=0.93 ---58305.15 ms/batch
Step 00018850 L_out=0.075 Acc=0.92 ---81900.30 ms/batch
Step 00018900 L_out=0.142 Acc=0.85 ---105722.07 ms/batch
Step 00018950 L_out=0.244 Acc=0.88 ---129131.04 ms/batch
Step 00019000 L_out=0.137 Acc=0.86 ---152888.14 ms/batch
Step 00019050 L_out=0.043 Acc=0.82 ---176661.43 ms/batch
Step 00019100 L_out=0.098 Acc=0.82 ---200386.24 ms/batch
Step 00019150 L_out=0.071 Acc=0.86 ---235161.96 ms/batch
Step 00019200 L_out=0.080 Acc=0.94 ---258930.77 ms/batch
Step 00019250 L_out=0.091 Acc=0.89 ---282551.47 ms/batch
Step 00019300 L_out=0.327 Acc=0.86 ---306447.95 ms/batch
Round 2 | epoch=28, train costTime=327.0064160823822, | total_activation_sum=12217495
Round 2 | epoch=29, current m_iou=0.6405808808269544
Round 2 | Best m_IoU is: 0.644, OA is: 0.906 | val costTime=117.54169607162476
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.150 Acc=0.85 --- 3367.38 ms/batch
Step 00019400 L_out=0.073 Acc=0.93 ---27552.38 ms/batch
Step 00019450 L_out=0.312 Acc=0.85 ---51176.71 ms/batch
Step 00019500 L_out=0.083 Acc=0.87 ---74953.15 ms/batch
Step 00019550 L_out=0.072 Acc=0.76 ---98740.53 ms/batch
Step 00019600 L_out=0.259 Acc=0.88 ---122692.40 ms/batch
Step 00019650 L_out=0.038 Acc=0.92 ---146759.64 ms/batch
Step 00019700 L_out=0.050 Acc=0.91 ---170599.48 ms/batch
Step 00019750 L_out=0.096 Acc=0.89 ---194330.70 ms/batch
Step 00019800 L_out=0.054 Acc=0.97 ---217970.71 ms/batch
Step 00019850 L_out=0.066 Acc=0.92 ---241809.54 ms/batch
Step 00019900 L_out=0.216 Acc=0.88 ---265568.22 ms/batch
Step 00019950 L_out=0.126 Acc=0.81 ---289466.06 ms/batch
Step 00020000 L_out=0.098 Acc=0.94 ---313153.70 ms/batch
Round 2 | epoch=29, train costTime=317.93828225135803, | total_activation_sum=12024122
Round 2 | epoch=30, current m_iou=0.6485290944848996
Round 2 | Best m_IoU is: 0.649, OA is: 0.908 | val costTime=117.86481952667236
Round 2 | ****EPOCH 30****
Round 3 | ****EPOCH 0****
Step 00020050 L_out=0.819 Acc=0.68 ---19011.58 ms/batch
Step 00020100 L_out=1.032 Acc=0.61 ---43016.96 ms/batch
Step 00020150 L_out=0.811 Acc=0.70 ---66729.88 ms/batch
Step 00020200 L_out=1.083 Acc=0.51 ---90600.65 ms/batch
Step 00020250 L_out=0.650 Acc=0.75 ---114257.76 ms/batch
Step 00020300 L_out=0.775 Acc=0.90 ---137968.44 ms/batch
Step 00020350 L_out=0.958 Acc=0.57 ---161795.32 ms/batch
Step 00020400 L_out=0.558 Acc=0.78 ---185381.69 ms/batch
Step 00020450 L_out=0.636 Acc=0.78 ---209311.85 ms/batch
Step 00020500 L_out=0.633 Acc=0.62 ---233100.77 ms/batch
Step 00020550 L_out=0.468 Acc=0.81 ---256867.84 ms/batch
Step 00020600 L_out=0.724 Acc=0.78 ---280557.86 ms/batch
Step 00020650 L_out=0.440 Acc=0.83 ---304261.12 ms/batch
Round 3 | epoch=0, train costTime=317.1943027973175, | total_activation_sum=22345644
Round 3 | ****EPOCH 1****
Step 00020700 L_out=0.586 Acc=0.83 ---11033.07 ms/batch
Step 00020750 L_out=0.547 Acc=0.85 ---34696.73 ms/batch
Step 00020800 L_out=0.829 Acc=0.81 ---58370.60 ms/batch
Step 00020850 L_out=0.250 Acc=0.88 ---82194.45 ms/batch
Step 00020900 L_out=0.414 Acc=0.82 ---105620.30 ms/batch
Step 00020950 L_out=0.314 Acc=0.84 ---129279.04 ms/batch
Step 00021000 L_out=0.374 Acc=0.93 ---152831.12 ms/batch
Step 00021050 L_out=0.882 Acc=0.77 ---176617.11 ms/batch
Step 00021100 L_out=1.062 Acc=0.80 ---200327.34 ms/batch
Step 00021150 L_out=0.471 Acc=0.86 ---224197.79 ms/batch
Step 00021200 L_out=0.484 Acc=0.88 ---247839.40 ms/batch
Step 00021250 L_out=0.460 Acc=0.93 ---271681.07 ms/batch
Step 00021300 L_out=0.399 Acc=0.77 ---295639.25 ms/batch
Round 3 | epoch=1, train costTime=316.5462851524353, | total_activation_sum=20680091
Round 3 | ****EPOCH 2****
Step 00021350 L_out=1.390 Acc=0.80 --- 2785.57 ms/batch
Step 00021400 L_out=0.511 Acc=0.83 ---26073.03 ms/batch
Step 00021450 L_out=0.310 Acc=0.83 ---49790.68 ms/batch
Step 00021500 L_out=0.242 Acc=0.88 ---73246.26 ms/batch
Step 00021550 L_out=0.638 Acc=0.77 ---96878.31 ms/batch
Step 00021600 L_out=0.602 Acc=0.80 ---120598.88 ms/batch
Step 00021650 L_out=0.393 Acc=0.91 ---144366.00 ms/batch
Step 00021700 L_out=0.498 Acc=0.57 ---168129.85 ms/batch
Step 00021750 L_out=0.424 Acc=0.84 ---192201.69 ms/batch
Step 00021800 L_out=0.386 Acc=0.88 ---215823.00 ms/batch
Step 00021850 L_out=0.514 Acc=0.88 ---239445.58 ms/batch
Step 00021900 L_out=0.826 Acc=0.77 ---262915.41 ms/batch
Step 00021950 L_out=0.435 Acc=0.85 ---286586.50 ms/batch
Step 00022000 L_out=0.742 Acc=0.80 ---310557.25 ms/batch
Round 3 | epoch=2, train costTime=315.7089350223541, | total_activation_sum=20799464
Round 3 | ****EPOCH 3****
Step 00022050 L_out=0.297 Acc=0.88 ---18576.77 ms/batch
Step 00022100 L_out=0.886 Acc=0.89 ---42122.31 ms/batch
Step 00022150 L_out=0.647 Acc=0.87 ---65733.00 ms/batch
Step 00022200 L_out=0.223 Acc=0.86 ---89213.42 ms/batch
Step 00022250 L_out=0.350 Acc=0.73 ---112933.12 ms/batch
Step 00022300 L_out=0.945 Acc=0.82 ---136660.87 ms/batch
Step 00022350 L_out=0.132 Acc=0.91 ---160199.98 ms/batch
Step 00022400 L_out=0.338 Acc=0.71 ---183935.13 ms/batch
Step 00022450 L_out=0.500 Acc=0.81 ---207584.47 ms/batch
Step 00022500 L_out=0.324 Acc=0.92 ---231327.82 ms/batch
Step 00022550 L_out=0.578 Acc=0.88 ---255239.26 ms/batch
Step 00022600 L_out=0.273 Acc=0.88 ---278971.96 ms/batch
Step 00022650 L_out=1.524 Acc=0.72 ---302896.82 ms/batch
Round 3 | epoch=3, train costTime=316.0836412906647, | total_activation_sum=20651361
Round 3 | ****EPOCH 4****
Step 00022700 L_out=0.228 Acc=0.93 ---10366.08 ms/batch
Step 00022750 L_out=0.231 Acc=0.80 ---34133.21 ms/batch
Step 00022800 L_out=0.413 Acc=0.90 ---58049.39 ms/batch
Step 00022850 L_out=0.404 Acc=0.88 ---81843.03 ms/batch
Step 00022900 L_out=1.238 Acc=0.74 ---105390.94 ms/batch
Step 00022950 L_out=0.421 Acc=0.83 ---129107.74 ms/batch
Step 00023000 L_out=0.509 Acc=0.87 ---153069.72 ms/batch
Step 00023050 L_out=0.236 Acc=0.87 ---176819.50 ms/batch
Step 00023100 L_out=0.254 Acc=0.89 ---200439.96 ms/batch
Step 00023150 L_out=0.321 Acc=0.87 ---224087.11 ms/batch
Step 00023200 L_out=0.331 Acc=0.90 ---247696.99 ms/batch
Step 00023250 L_out=0.294 Acc=0.89 ---271464.45 ms/batch
Step 00023300 L_out=0.562 Acc=0.82 ---295102.16 ms/batch
Round 3 | epoch=4, train costTime=316.3140423297882, | total_activation_sum=20705443
Round 3 | ****EPOCH 5****
Step 00023350 L_out=0.132 Acc=0.91 --- 2379.12 ms/batch
Step 00023400 L_out=0.798 Acc=0.88 ---25912.84 ms/batch
Step 00023450 L_out=0.682 Acc=0.85 ---49579.48 ms/batch
Step 00023500 L_out=0.260 Acc=0.90 ---73201.48 ms/batch
Step 00023550 L_out=0.724 Acc=0.84 ---96975.16 ms/batch
Step 00023600 L_out=0.222 Acc=0.91 ---120811.05 ms/batch
Step 00023650 L_out=0.138 Acc=0.90 ---144391.31 ms/batch
Step 00023700 L_out=0.468 Acc=0.79 ---168129.60 ms/batch
Step 00023750 L_out=0.538 Acc=0.80 ---191660.93 ms/batch
Step 00023800 L_out=0.389 Acc=0.76 ---215345.14 ms/batch
Step 00023850 L_out=0.496 Acc=0.76 ---238864.89 ms/batch
Step 00023900 L_out=0.475 Acc=0.85 ---262391.34 ms/batch
Step 00023950 L_out=0.387 Acc=0.88 ---286148.33 ms/batch
Step 00024000 L_out=0.172 Acc=0.89 ---309778.66 ms/batch
Round 3 | epoch=5, train costTime=315.5022556781769, | total_activation_sum=20790338
Round 3 | ****EPOCH 6****
Step 00024050 L_out=0.346 Acc=0.80 ---17917.65 ms/batch
Step 00024100 L_out=0.330 Acc=0.82 ---40487.04 ms/batch
Step 00024150 L_out=0.513 Acc=0.90 ---63682.51 ms/batch
Step 00024200 L_out=0.724 Acc=0.71 ---87133.99 ms/batch
Step 00024250 L_out=0.494 Acc=0.93 ---110548.83 ms/batch
Step 00024300 L_out=0.250 Acc=0.89 ---133936.56 ms/batch
Step 00024350 L_out=0.602 Acc=0.79 ---207187.51 ms/batch
Step 00024400 L_out=0.327 Acc=0.86 ---230754.20 ms/batch
Step 00024450 L_out=0.230 Acc=0.93 ---254334.29 ms/batch
Step 00024500 L_out=0.426 Acc=0.91 ---277072.35 ms/batch
Step 00024550 L_out=0.271 Acc=0.84 ---299803.84 ms/batch
Step 00024600 L_out=0.579 Acc=0.94 ---323226.00 ms/batch
Step 00024650 L_out=0.402 Acc=0.92 ---346724.60 ms/batch
Round 3 | epoch=6, train costTime=360.08618783950806, | total_activation_sum=20842242
Round 3 | ****EPOCH 7****
Step 00024700 L_out=0.253 Acc=0.91 --- 9869.20 ms/batch
Step 00024750 L_out=0.281 Acc=0.89 ---48326.17 ms/batch
Step 00024800 L_out=0.302 Acc=0.94 ---72266.41 ms/batch
Step 00024850 L_out=0.317 Acc=0.87 ---96371.05 ms/batch
Step 00024900 L_out=0.332 Acc=0.90 ---119975.68 ms/batch
Step 00024950 L_out=0.482 Acc=0.92 ---143748.39 ms/batch
Step 00025000 L_out=0.157 Acc=0.89 ---167530.72 ms/batch
Step 00025050 L_out=0.336 Acc=0.92 ---191022.34 ms/batch
Step 00025100 L_out=0.247 Acc=0.91 ---214714.40 ms/batch
Step 00025150 L_out=0.421 Acc=0.88 ---238360.08 ms/batch
Step 00025200 L_out=0.305 Acc=0.83 ---262148.86 ms/batch
Step 00025250 L_out=0.302 Acc=0.92 ---285703.77 ms/batch
Step 00025300 L_out=0.263 Acc=0.90 ---309273.70 ms/batch
Round 3 | epoch=7, train costTime=331.124632358551, | total_activation_sum=20566047
Round 3 | ****EPOCH 8****
Step 00025350 L_out=0.307 Acc=0.89 --- 1932.07 ms/batch
Step 00025400 L_out=0.418 Acc=0.78 ---25797.36 ms/batch
Step 00025450 L_out=0.145 Acc=0.95 ---49575.34 ms/batch
Step 00025500 L_out=0.990 Acc=0.84 ---73271.64 ms/batch
Step 00025550 L_out=0.302 Acc=0.87 ---96759.15 ms/batch
Step 00025600 L_out=0.508 Acc=0.89 ---120420.57 ms/batch
Step 00025650 L_out=0.300 Acc=0.91 ---144261.87 ms/batch
Step 00025700 L_out=0.256 Acc=0.91 ---168033.38 ms/batch
Step 00025750 L_out=0.204 Acc=0.87 ---191705.73 ms/batch
Step 00025800 L_out=0.215 Acc=0.85 ---215458.11 ms/batch
Step 00025850 L_out=0.354 Acc=0.80 ---239103.94 ms/batch
Step 00025900 L_out=0.621 Acc=0.87 ---262666.79 ms/batch
Step 00025950 L_out=0.327 Acc=0.92 ---286203.20 ms/batch
Step 00026000 L_out=0.465 Acc=0.87 ---309928.81 ms/batch
Round 3 | epoch=8, train costTime=316.19196486473083, | total_activation_sum=20700079
Round 3 | ****EPOCH 9****
Step 00026050 L_out=0.351 Acc=0.85 ---17438.92 ms/batch
Step 00026100 L_out=0.503 Acc=0.83 ---41051.16 ms/batch
Step 00026150 L_out=0.401 Acc=0.91 ---64901.79 ms/batch
Step 00026200 L_out=0.627 Acc=0.88 ---89058.87 ms/batch
Step 00026250 L_out=0.294 Acc=0.91 ---112861.25 ms/batch
Step 00026300 L_out=0.374 Acc=0.92 ---136415.39 ms/batch
Step 00026350 L_out=0.235 Acc=0.93 ---159968.37 ms/batch
Step 00026400 L_out=0.316 Acc=0.89 ---183838.30 ms/batch
Step 00026450 L_out=0.249 Acc=0.94 ---207489.31 ms/batch
Step 00026500 L_out=0.385 Acc=0.86 ---231156.93 ms/batch
Step 00026550 L_out=0.223 Acc=0.92 ---254898.15 ms/batch
Step 00026600 L_out=0.223 Acc=0.91 ---278716.06 ms/batch
Step 00026650 L_out=0.450 Acc=0.88 ---302377.96 ms/batch
Round 3 | epoch=9, train costTime=316.55664324760437, | total_activation_sum=20690530
Round 3 | ****EPOCH 10****
Step 00026700 L_out=0.206 Acc=0.87 --- 9531.07 ms/batch
Step 00026750 L_out=0.389 Acc=0.89 ---33236.10 ms/batch
Step 00026800 L_out=0.199 Acc=0.96 ---57135.63 ms/batch
Step 00026850 L_out=0.863 Acc=0.80 ---80668.97 ms/batch
Step 00026900 L_out=0.183 Acc=0.89 ---104281.13 ms/batch
Step 00026950 L_out=0.653 Acc=0.85 ---127935.25 ms/batch
Step 00027000 L_out=0.202 Acc=0.90 ---151794.04 ms/batch
Step 00027050 L_out=0.171 Acc=0.91 ---175749.40 ms/batch
Step 00027100 L_out=0.444 Acc=0.82 ---199477.03 ms/batch
Step 00027150 L_out=0.230 Acc=0.92 ---223284.94 ms/batch
Step 00027200 L_out=0.175 Acc=0.89 ---246981.16 ms/batch
Step 00027250 L_out=0.445 Acc=0.93 ---270699.89 ms/batch
Step 00027300 L_out=0.893 Acc=0.71 ---294554.87 ms/batch
Round 3 | epoch=10, train costTime=317.03368759155273, | total_activation_sum=20824511
Round 3 | ****EPOCH 11****
Step 00027350 L_out=0.418 Acc=0.85 --- 1448.93 ms/batch
Step 00027400 L_out=0.076 Acc=0.96 ---25102.83 ms/batch
Step 00027450 L_out=0.103 Acc=0.95 ---48667.01 ms/batch
Step 00027500 L_out=0.202 Acc=0.84 ---72216.98 ms/batch
Step 00027550 L_out=0.457 Acc=0.87 ---95809.00 ms/batch
Step 00027600 L_out=0.151 Acc=0.90 ---119399.67 ms/batch
Step 00027650 L_out=0.456 Acc=0.87 ---142837.36 ms/batch
Step 00027700 L_out=0.218 Acc=0.81 ---166438.08 ms/batch
Step 00027750 L_out=0.216 Acc=0.91 ---190048.32 ms/batch
Step 00027800 L_out=0.302 Acc=0.82 ---213637.04 ms/batch
Step 00027850 L_out=0.218 Acc=0.90 ---237268.13 ms/batch
Step 00027900 L_out=0.215 Acc=0.89 ---260844.83 ms/batch
Step 00027950 L_out=0.922 Acc=0.90 ---284510.09 ms/batch
Step 00028000 L_out=0.267 Acc=0.90 ---308348.50 ms/batch
Round 3 | epoch=11, train costTime=315.0123391151428, | total_activation_sum=20929586
Round 3 | ****EPOCH 12****
Step 00028050 L_out=0.190 Acc=0.96 ---17159.02 ms/batch
Step 00028100 L_out=0.196 Acc=0.94 ---40663.23 ms/batch
Step 00028150 L_out=0.098 Acc=0.93 ---64203.69 ms/batch
Step 00028200 L_out=0.238 Acc=0.90 ---87933.62 ms/batch
Step 00028250 L_out=0.385 Acc=0.90 ---111846.77 ms/batch
Step 00028300 L_out=0.101 Acc=0.94 ---135590.26 ms/batch
Step 00028350 L_out=0.190 Acc=0.88 ---159185.13 ms/batch
Step 00028400 L_out=0.533 Acc=0.81 ---182970.40 ms/batch
Step 00028450 L_out=0.306 Acc=0.83 ---206673.39 ms/batch
Step 00028500 L_out=0.182 Acc=0.92 ---230472.49 ms/batch
Step 00028550 L_out=0.251 Acc=0.89 ---254256.20 ms/batch
Step 00028600 L_out=0.259 Acc=0.84 ---277854.94 ms/batch
Step 00028650 L_out=0.127 Acc=0.94 ---301433.58 ms/batch
Round 3 | epoch=12, train costTime=316.2115340232849, | total_activation_sum=20697540
Round 3 | ****EPOCH 13****
Step 00028700 L_out=0.554 Acc=0.81 --- 9100.77 ms/batch
Step 00028750 L_out=0.274 Acc=0.86 ---32886.61 ms/batch
Step 00028800 L_out=0.222 Acc=0.89 ---56384.33 ms/batch
Step 00028850 L_out=0.250 Acc=0.90 ---80239.22 ms/batch
Step 00028900 L_out=0.097 Acc=0.96 ---103695.48 ms/batch
Step 00028950 L_out=0.252 Acc=0.93 ---127500.52 ms/batch
Step 00029000 L_out=0.110 Acc=0.95 ---151257.19 ms/batch
Step 00029050 L_out=0.297 Acc=0.85 ---174878.15 ms/batch
Step 00029100 L_out=0.453 Acc=0.89 ---198615.66 ms/batch
Step 00029150 L_out=0.142 Acc=0.90 ---222192.27 ms/batch
Step 00029200 L_out=0.639 Acc=0.85 ---245730.48 ms/batch
Step 00029250 L_out=0.487 Acc=0.85 ---269396.24 ms/batch
Step 00029300 L_out=0.175 Acc=0.90 ---292881.49 ms/batch
Round 3 | epoch=13, train costTime=315.6223978996277, | total_activation_sum=20807969
Round 3 | ****EPOCH 14****
Step 00029350 L_out=0.183 Acc=0.94 ---  941.56 ms/batch
Step 00029400 L_out=0.415 Acc=0.86 ---24562.69 ms/batch
Step 00029450 L_out=0.106 Acc=0.88 ---48296.54 ms/batch
Step 00029500 L_out=0.241 Acc=0.89 ---72066.57 ms/batch
Step 00029550 L_out=0.264 Acc=0.92 ---95901.69 ms/batch
Step 00029600 L_out=0.209 Acc=0.86 ---119429.83 ms/batch
Step 00029650 L_out=0.138 Acc=0.93 ---143049.20 ms/batch
Step 00029700 L_out=0.200 Acc=0.96 ---166882.03 ms/batch
Step 00029750 L_out=0.304 Acc=0.88 ---190568.91 ms/batch
Step 00029800 L_out=0.170 Acc=0.93 ---214393.80 ms/batch
Step 00029850 L_out=0.145 Acc=0.88 ---237940.67 ms/batch
Step 00029900 L_out=0.213 Acc=0.91 ---261591.28 ms/batch
Step 00029950 L_out=0.198 Acc=0.95 ---285182.84 ms/batch
Step 00030000 L_out=0.209 Acc=0.92 ---309049.03 ms/batch
Round 3 | epoch=14, train costTime=316.24160385131836, | total_activation_sum=20667861
Round 3 | ****EPOCH 15****
Step 00030050 L_out=0.532 Acc=0.81 ---16566.34 ms/batch
Step 00030100 L_out=0.248 Acc=0.93 ---40229.71 ms/batch
Step 00030150 L_out=0.388 Acc=0.85 ---63776.55 ms/batch
Step 00030200 L_out=0.235 Acc=0.90 ---87439.37 ms/batch
Step 00030250 L_out=0.203 Acc=0.95 ---111124.61 ms/batch
Step 00030300 L_out=0.163 Acc=0.89 ---134625.12 ms/batch
Step 00030350 L_out=0.203 Acc=0.90 ---158197.33 ms/batch
Step 00030400 L_out=0.203 Acc=0.92 ---181858.03 ms/batch
Step 00030450 L_out=0.264 Acc=0.93 ---205465.65 ms/batch
Step 00030500 L_out=0.236 Acc=0.94 ---229060.34 ms/batch
Step 00030550 L_out=0.181 Acc=0.90 ---252700.76 ms/batch
Step 00030600 L_out=0.594 Acc=0.85 ---276285.25 ms/batch
Step 00030650 L_out=0.163 Acc=0.85 ---300055.60 ms/batch
Round 3 | epoch=15, train costTime=315.3844120502472, | total_activation_sum=20741718
Round 3 | ****EPOCH 16****
Step 00030700 L_out=0.119 Acc=0.95 --- 8610.77 ms/batch
Step 00030750 L_out=0.349 Acc=0.88 ---32284.72 ms/batch
Step 00030800 L_out=0.194 Acc=0.91 ---56001.53 ms/batch
Step 00030850 L_out=0.055 Acc=0.94 ---79776.38 ms/batch
Step 00030900 L_out=1.457 Acc=0.89 ---103383.12 ms/batch
Step 00030950 L_out=0.274 Acc=0.90 ---127085.59 ms/batch
Step 00031000 L_out=0.480 Acc=0.86 ---150859.22 ms/batch
Step 00031050 L_out=0.077 Acc=0.96 ---174592.02 ms/batch
Step 00031100 L_out=0.337 Acc=0.89 ---198217.79 ms/batch
Step 00031150 L_out=0.188 Acc=0.91 ---221929.77 ms/batch
Step 00031200 L_out=0.262 Acc=0.95 ---245487.02 ms/batch
Step 00031250 L_out=0.153 Acc=0.92 ---269346.96 ms/batch
Step 00031300 L_out=0.608 Acc=0.84 ---292989.44 ms/batch
Round 3 | epoch=16, train costTime=316.18001556396484, | total_activation_sum=20888185
Round 3 | ****EPOCH 17****
Step 00031350 L_out=0.212 Acc=0.95 ---  465.76 ms/batch
Step 00031400 L_out=0.170 Acc=0.93 ---23871.50 ms/batch
Step 00031450 L_out=0.261 Acc=0.89 ---47527.21 ms/batch
Step 00031500 L_out=0.198 Acc=0.93 ---71198.52 ms/batch
Step 00031550 L_out=0.401 Acc=0.90 ---94782.63 ms/batch
Step 00031600 L_out=0.201 Acc=0.87 ---118554.04 ms/batch
Step 00031650 L_out=0.189 Acc=0.89 ---142290.77 ms/batch
Step 00031700 L_out=0.300 Acc=0.87 ---166064.68 ms/batch
Step 00031750 L_out=0.082 Acc=0.97 ---189660.28 ms/batch
Step 00031800 L_out=0.066 Acc=0.91 ---213260.36 ms/batch
Step 00031850 L_out=0.270 Acc=0.96 ---237391.41 ms/batch
Step 00031900 L_out=0.182 Acc=0.95 ---261228.57 ms/batch
Round 2 | ****EPOCH 0****
Step 00000050 L_out=1.190 Acc=0.37 ---25454.63 ms/batch
Step 00000100 L_out=2.285 Acc=0.40 ---49130.72 ms/batch
Step 00000150 L_out=0.407 Acc=0.64 ---72959.54 ms/batch
Step 00000200 L_out=1.153 Acc=0.07 ---96655.32 ms/batch
Step 00000250 L_out=0.757 Acc=0.41 ---120297.67 ms/batch
Step 00000300 L_out=0.376 Acc=0.46 ---144129.42 ms/batch
Step 00000350 L_out=0.431 Acc=0.31 ---167976.59 ms/batch
Step 00000400 L_out=0.659 Acc=0.47 ---191753.75 ms/batch
Step 00000450 L_out=1.024 Acc=0.47 ---215482.96 ms/batch
Step 00000500 L_out=0.876 Acc=0.41 ---239257.93 ms/batch
Step 00000550 L_out=1.084 Acc=0.47 ---262743.60 ms/batch
Step 00000600 L_out=0.808 Acc=0.49 ---286495.17 ms/batch
Step 00000650 L_out=0.588 Acc=0.46 ---310272.10 ms/batch
Round 2 | epoch=0, train costTime=318.3142375946045, | total_activation_sum=14610914
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.413 Acc=0.32 ---15522.13 ms/batch
Step 00000750 L_out=0.831 Acc=0.50 ---39268.96 ms/batch
Step 00000800 L_out=1.177 Acc=0.30 ---62873.09 ms/batch
Step 00000850 L_out=0.745 Acc=0.46 ---86632.85 ms/batch
Step 00000900 L_out=0.396 Acc=0.26 ---110341.30 ms/batch
Step 00000950 L_out=0.378 Acc=0.65 ---134371.49 ms/batch
Step 00001000 L_out=0.674 Acc=0.58 ---158227.42 ms/batch
Step 00001050 L_out=0.855 Acc=0.37 ---181731.84 ms/batch
Step 00001100 L_out=0.466 Acc=0.32 ---205432.75 ms/batch
Step 00001150 L_out=0.803 Acc=0.56 ---229287.25 ms/batch
Step 00001200 L_out=0.409 Acc=0.71 ---253030.72 ms/batch
Step 00001250 L_out=0.483 Acc=0.67 ---276559.64 ms/batch
Step 00001300 L_out=0.449 Acc=0.49 ---300318.40 ms/batch
Round 2 | epoch=1, train costTime=316.362788438797, | total_activation_sum=12956350
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.669 Acc=0.57 --- 7582.41 ms/batch
Step 00001400 L_out=0.263 Acc=0.52 ---31335.80 ms/batch
Step 00001450 L_out=0.453 Acc=0.41 ---54849.16 ms/batch
Step 00001500 L_out=0.509 Acc=0.57 ---78393.34 ms/batch
Step 00001550 L_out=1.151 Acc=0.77 ---101816.48 ms/batch
Step 00001600 L_out=0.353 Acc=0.76 ---125568.31 ms/batch
Step 00001650 L_out=0.797 Acc=0.47 ---149114.11 ms/batch
Step 00001700 L_out=0.867 Acc=0.54 ---172951.92 ms/batch
Step 00001750 L_out=0.648 Acc=0.51 ---196550.21 ms/batch
Step 00001800 L_out=0.328 Acc=0.67 ---220150.57 ms/batch
Step 00001850 L_out=0.453 Acc=0.55 ---243812.96 ms/batch
Step 00001900 L_out=0.189 Acc=0.62 ---267439.15 ms/batch
Step 00001950 L_out=0.346 Acc=0.63 ---291345.92 ms/batch
Step 00002000 L_out=0.994 Acc=0.72 ---314911.95 ms/batch
Round 2 | epoch=2, train costTime=315.3797128200531, | total_activation_sum=12885538
Round 2 | ****EPOCH 3****
Step 00002050 L_out=0.511 Acc=0.59 ---23021.03 ms/batch
Step 00002100 L_out=0.620 Acc=0.65 ---46843.51 ms/batch
Step 00002150 L_out=0.546 Acc=0.63 ---70488.53 ms/batch
Step 00002200 L_out=0.783 Acc=0.66 ---94325.30 ms/batch
Step 00002250 L_out=0.342 Acc=0.48 ---117804.07 ms/batch
Step 00002300 L_out=0.601 Acc=0.66 ---141498.58 ms/batch
Step 00002350 L_out=0.501 Acc=0.81 ---165312.56 ms/batch
Step 00002400 L_out=0.231 Acc=0.54 ---188951.54 ms/batch
Step 00002450 L_out=0.543 Acc=0.82 ---212657.97 ms/batch
Step 00002500 L_out=0.776 Acc=0.57 ---236362.42 ms/batch
Step 00002550 L_out=0.544 Acc=0.75 ---259848.25 ms/batch
Step 00002600 L_out=0.176 Acc=0.62 ---283451.18 ms/batch
Step 00002650 L_out=0.299 Acc=0.52 ---307061.80 ms/batch
Round 2 | epoch=3, train costTime=315.61622381210327, | total_activation_sum=12974073
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.347 Acc=0.73 ---15198.89 ms/batch
Step 00002750 L_out=0.557 Acc=0.64 ---38646.04 ms/batch
Step 00002800 L_out=0.474 Acc=0.60 ---62260.72 ms/batch
Step 00002850 L_out=0.446 Acc=0.85 ---86040.32 ms/batch
Step 00002900 L_out=0.394 Acc=0.67 ---109560.05 ms/batch
Step 00002950 L_out=2.223 Acc=0.74 ---133034.93 ms/batch
Step 00003000 L_out=0.550 Acc=0.62 ---156701.94 ms/batch
Step 00003050 L_out=1.057 Acc=0.62 ---180580.01 ms/batch
Step 00003100 L_out=0.323 Acc=0.73 ---204172.38 ms/batch
Step 00003150 L_out=0.203 Acc=0.80 ---228045.60 ms/batch
Step 00003200 L_out=0.432 Acc=0.60 ---251593.18 ms/batch
Step 00003250 L_out=0.556 Acc=0.63 ---275373.79 ms/batch
Step 00003300 L_out=0.216 Acc=0.87 ---299010.08 ms/batch
Round 2 | epoch=4, train costTime=315.51983404159546, | total_activation_sum=12853818
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.397 Acc=0.71 --- 7069.71 ms/batch
Step 00003400 L_out=0.448 Acc=0.60 ---30631.27 ms/batch
Step 00003450 L_out=0.553 Acc=0.54 ---54397.92 ms/batch
Step 00003500 L_out=0.190 Acc=0.76 ---78245.84 ms/batch
Step 00003550 L_out=0.275 Acc=0.68 ---101937.46 ms/batch
Step 00003600 L_out=2.120 Acc=0.47 ---125482.48 ms/batch
Step 00003650 L_out=0.324 Acc=0.52 ---149096.60 ms/batch
Step 00003700 L_out=0.469 Acc=0.74 ---172858.73 ms/batch
Step 00003750 L_out=0.455 Acc=0.72 ---196321.56 ms/batch
Step 00003800 L_out=0.365 Acc=0.74 ---220046.98 ms/batch
Step 00003850 L_out=0.181 Acc=0.75 ---243722.32 ms/batch
Step 00003900 L_out=0.387 Acc=0.64 ---267411.17 ms/batch
Step 00003950 L_out=0.304 Acc=0.74 ---291283.52 ms/batch
Step 00004000 L_out=0.571 Acc=0.73 ---314795.12 ms/batch
Round 2 | epoch=5, train costTime=315.7406885623932, | total_activation_sum=12992702
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.237 Acc=0.85 ---22699.36 ms/batch
Step 00004100 L_out=0.290 Acc=0.80 ---46338.97 ms/batch
Step 00004150 L_out=0.626 Acc=0.69 ---69945.01 ms/batch
Step 00004200 L_out=0.352 Acc=0.48 ---93703.88 ms/batch
Step 00004250 L_out=0.343 Acc=0.84 ---117394.10 ms/batch
Step 00004300 L_out=0.158 Acc=0.74 ---141083.77 ms/batch
Step 00004350 L_out=0.269 Acc=0.72 ---164888.49 ms/batch
Step 00004400 L_out=0.950 Acc=0.55 ---188500.52 ms/batch
Step 00004450 L_out=0.590 Acc=0.83 ---211995.49 ms/batch
Step 00004500 L_out=0.307 Acc=0.76 ---235872.15 ms/batch
Step 00004550 L_out=0.150 Acc=0.71 ---259605.55 ms/batch
Step 00004600 L_out=0.288 Acc=0.75 ---283133.56 ms/batch
Step 00004650 L_out=0.531 Acc=0.82 ---306840.58 ms/batch
Round 2 | epoch=6, train costTime=315.85843110084534, | total_activation_sum=13056591
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.332 Acc=0.80 ---14600.33 ms/batch
Step 00004750 L_out=0.306 Acc=0.70 ---38189.04 ms/batch
Step 00004800 L_out=0.352 Acc=0.77 ---61932.75 ms/batch
Step 00004850 L_out=0.144 Acc=0.76 ---85852.49 ms/batch
Step 00004900 L_out=0.259 Acc=0.74 ---109480.10 ms/batch
Step 00004950 L_out=0.105 Acc=0.71 ---133523.12 ms/batch
Step 00005000 L_out=0.119 Acc=0.76 ---157118.29 ms/batch
Step 00005050 L_out=0.115 Acc=0.71 ---180749.94 ms/batch
Step 00005100 L_out=0.799 Acc=0.82 ---204618.66 ms/batch
Step 00005150 L_out=0.181 Acc=0.80 ---228185.60 ms/batch
Step 00005200 L_out=0.601 Acc=0.72 ---251612.39 ms/batch
Step 00005250 L_out=0.502 Acc=0.69 ---275296.46 ms/batch
Step 00005300 L_out=0.457 Acc=0.61 ---299149.99 ms/batch
Round 2 | epoch=7, train costTime=316.17682337760925, | total_activation_sum=12986343
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.329 Acc=0.82 --- 6497.44 ms/batch
Step 00005400 L_out=0.436 Acc=0.66 ---29983.58 ms/batch
Step 00005450 L_out=0.234 Acc=0.64 ---53817.21 ms/batch
Step 00005500 L_out=0.103 Acc=0.62 ---77373.73 ms/batch
Step 00005550 L_out=0.370 Acc=0.77 ---101018.68 ms/batch
Step 00005600 L_out=0.195 Acc=0.89 ---124631.97 ms/batch
Step 00005650 L_out=0.242 Acc=0.75 ---148346.13 ms/batch
Step 00005700 L_out=0.231 Acc=0.83 ---172045.07 ms/batch
Step 00005750 L_out=0.889 Acc=0.49 ---195674.18 ms/batch
Step 00005800 L_out=0.179 Acc=0.82 ---219335.57 ms/batch
Step 00005850 L_out=0.990 Acc=0.84 ---242979.31 ms/batch
Step 00005900 L_out=0.266 Acc=0.82 ---266665.95 ms/batch
Step 00005950 L_out=0.813 Acc=0.67 ---290601.59 ms/batch
Step 00006000 L_out=0.358 Acc=0.79 ---314273.48 ms/batch
Round 2 | epoch=8, train costTime=315.70124530792236, | total_activation_sum=12967529
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.177 Acc=0.75 ---22263.76 ms/batch
Step 00006100 L_out=0.150 Acc=0.71 ---46092.64 ms/batch
Step 00006150 L_out=0.287 Acc=0.70 ---69830.60 ms/batch
Step 00006200 L_out=0.463 Acc=0.71 ---93388.17 ms/batch
Step 00006250 L_out=0.224 Acc=0.62 ---116753.69 ms/batch
Step 00006300 L_out=0.379 Acc=0.70 ---140564.27 ms/batch
Step 00006350 L_out=0.252 Acc=0.75 ---164126.06 ms/batch
Step 00006400 L_out=0.426 Acc=0.83 ---187864.43 ms/batch
Step 00006450 L_out=0.198 Acc=0.83 ---211744.56 ms/batch
Step 00006500 L_out=0.320 Acc=0.85 ---235294.33 ms/batch
Step 00006550 L_out=0.690 Acc=0.60 ---258940.48 ms/batch
Step 00006600 L_out=0.518 Acc=0.71 ---282628.62 ms/batch
Step 00006650 L_out=0.292 Acc=0.82 ---306261.74 ms/batch
Round 2 | epoch=9, train costTime=315.8225402832031, | total_activation_sum=12962116
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.112 Acc=0.57 ---14081.20 ms/batch
Step 00006750 L_out=0.249 Acc=0.73 ---37696.69 ms/batch
Step 00006800 L_out=0.403 Acc=0.84 ---61186.06 ms/batch
Step 00006850 L_out=0.267 Acc=0.83 ---84777.35 ms/batch
Step 00006900 L_out=0.340 Acc=0.78 ---108456.81 ms/batch
Step 00006950 L_out=1.125 Acc=0.63 ---132083.53 ms/batch
Step 00007000 L_out=0.599 Acc=0.79 ---155714.03 ms/batch
Step 00007050 L_out=0.393 Acc=0.72 ---179544.47 ms/batch
Step 00007100 L_out=0.151 Acc=0.73 ---203201.68 ms/batch
Step 00007150 L_out=0.516 Acc=0.81 ---226815.89 ms/batch
Step 00007200 L_out=0.562 Acc=0.65 ---250714.01 ms/batch
Step 00007250 L_out=0.344 Acc=0.73 ---274228.63 ms/batch
Step 00007300 L_out=0.172 Acc=0.84 ---297781.77 ms/batch
Round 2 | epoch=10, train costTime=315.31959414482117, | total_activation_sum=13043444
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.139 Acc=0.73 --- 6126.52 ms/batch
Step 00007400 L_out=0.192 Acc=0.84 ---29716.81 ms/batch
Step 00007450 L_out=0.263 Acc=0.75 ---53381.07 ms/batch
Step 00007500 L_out=0.707 Acc=0.70 ---77216.77 ms/batch
Step 00007550 L_out=0.214 Acc=0.88 ---100925.86 ms/batch
Step 00007600 L_out=0.149 Acc=0.74 ---124504.74 ms/batch
Step 00007650 L_out=1.475 Acc=0.85 ---148316.07 ms/batch
Step 00007700 L_out=0.575 Acc=0.66 ---172062.79 ms/batch
Step 00007750 L_out=0.453 Acc=0.60 ---195695.70 ms/batch
Step 00007800 L_out=0.175 Acc=0.75 ---219525.97 ms/batch
Step 00007850 L_out=0.219 Acc=0.68 ---243090.80 ms/batch
Step 00007900 L_out=0.626 Acc=0.80 ---266614.42 ms/batch
Step 00007950 L_out=0.253 Acc=0.75 ---290184.57 ms/batch
Step 00008000 L_out=0.316 Acc=0.72 ---313859.04 ms/batch
Round 2 | epoch=11, train costTime=315.78080129623413, | total_activation_sum=13065172
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.203 Acc=0.82 ---21720.65 ms/batch
Step 00008100 L_out=0.255 Acc=0.82 ---45215.71 ms/batch
Step 00008150 L_out=0.119 Acc=0.83 ---68998.27 ms/batch
Step 00008200 L_out=0.250 Acc=0.75 ---92421.59 ms/batch
Step 00008250 L_out=0.232 Acc=0.73 ---115978.05 ms/batch
Step 00008300 L_out=0.311 Acc=0.76 ---139582.24 ms/batch
Step 00008350 L_out=0.212 Acc=0.74 ---163277.72 ms/batch
Step 00008400 L_out=0.170 Acc=0.86 ---186884.65 ms/batch
Step 00008450 L_out=0.209 Acc=0.85 ---210412.37 ms/batch
Step 00008500 L_out=0.327 Acc=0.79 ---233898.76 ms/batch
Step 00008550 L_out=0.869 Acc=0.89 ---257555.93 ms/batch
Step 00008600 L_out=0.185 Acc=0.79 ---281276.69 ms/batch
Step 00008650 L_out=0.980 Acc=0.53 ---305168.09 ms/batch
Round 2 | epoch=12, train costTime=315.0267496109009, | total_activation_sum=12947046
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.276 Acc=0.84 ---13722.99 ms/batch
Step 00008750 L_out=0.159 Acc=0.83 ---37470.54 ms/batch
Step 00008800 L_out=0.235 Acc=0.88 ---61063.33 ms/batch
Step 00008850 L_out=0.149 Acc=0.73 ---84619.01 ms/batch
Step 00008900 L_out=0.447 Acc=0.86 ---108161.58 ms/batch
Step 00008950 L_out=0.142 Acc=0.90 ---131949.45 ms/batch
Step 00009000 L_out=0.325 Acc=0.81 ---155449.29 ms/batch
Step 00009050 L_out=0.183 Acc=0.80 ---179259.69 ms/batch
Step 00009100 L_out=0.870 Acc=0.81 ---202994.59 ms/batch
Step 00009150 L_out=0.178 Acc=0.85 ---226714.97 ms/batch
Step 00009200 L_out=0.095 Acc=0.72 ---250169.23 ms/batch
Step 00009250 L_out=0.187 Acc=0.76 ---273797.18 ms/batch
Step 00009300 L_out=0.237 Acc=0.87 ---297529.00 ms/batch
Round 2 | epoch=13, train costTime=315.53673553466797, | total_activation_sum=12894252
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.271 Acc=0.85 --- 5638.29 ms/batch
Step 00009400 L_out=0.126 Acc=0.90 ---29303.96 ms/batch
Step 00009450 L_out=0.114 Acc=0.69 ---52998.38 ms/batch
Step 00009500 L_out=0.200 Acc=0.76 ---76812.69 ms/batch
Step 00009550 L_out=0.331 Acc=0.83 ---100463.39 ms/batch
Step 00009600 L_out=0.221 Acc=0.81 ---124282.28 ms/batch
Step 00009650 L_out=0.478 Acc=0.77 ---147860.80 ms/batch
Step 00009700 L_out=0.241 Acc=0.83 ---171548.56 ms/batch
Step 00009750 L_out=0.201 Acc=0.81 ---195294.11 ms/batch
Step 00009800 L_out=0.261 Acc=0.61 ---218866.47 ms/batch
Step 00009850 L_out=0.378 Acc=0.72 ---242474.75 ms/batch
Step 00009900 L_out=0.477 Acc=0.84 ---266180.22 ms/batch
Step 00009950 L_out=0.611 Acc=0.75 ---289818.69 ms/batch
Step 00010000 L_out=0.273 Acc=0.86 ---313363.05 ms/batch
Round 2 | epoch=14, train costTime=315.76779890060425, | total_activation_sum=12970278
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.400 Acc=0.90 ---21258.64 ms/batch
Step 00010100 L_out=0.192 Acc=0.90 ---44920.84 ms/batch
Step 00010150 L_out=0.267 Acc=0.87 ---68503.74 ms/batch
Step 00010200 L_out=0.229 Acc=0.88 ---92282.84 ms/batch
Step 00010250 L_out=0.293 Acc=0.85 ---115821.15 ms/batch
Step 00010300 L_out=0.241 Acc=0.85 ---139632.20 ms/batch
Step 00010350 L_out=0.258 Acc=0.83 ---163303.81 ms/batch
Step 00010400 L_out=0.637 Acc=0.83 ---187054.01 ms/batch
Step 00010450 L_out=0.374 Acc=0.85 ---210490.13 ms/batch
Step 00010500 L_out=0.319 Acc=0.83 ---234063.70 ms/batch
Step 00010550 L_out=0.045 Acc=0.82 ---257707.93 ms/batch
Step 00010600 L_out=0.589 Acc=0.85 ---281355.19 ms/batch
Step 00010650 L_out=0.172 Acc=0.82 ---305058.92 ms/batch
Round 2 | epoch=15, train costTime=315.4358186721802, | total_activation_sum=13044334
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.082 Acc=0.86 ---13423.75 ms/batch
Step 00010750 L_out=0.332 Acc=0.74 ---37070.73 ms/batch
Step 00010800 L_out=0.152 Acc=0.92 ---60719.17 ms/batch
Step 00010850 L_out=0.118 Acc=0.73 ---84412.49 ms/batch
Step 00010900 L_out=0.281 Acc=0.84 ---107964.06 ms/batch
Step 00010950 L_out=0.325 Acc=0.81 ---131451.32 ms/batch
Step 00011000 L_out=0.414 Acc=0.88 ---155160.57 ms/batch
Step 00011050 L_out=0.441 Acc=0.79 ---178713.48 ms/batch
Step 00011100 L_out=0.092 Acc=0.87 ---202267.36 ms/batch
Step 00011150 L_out=0.408 Acc=0.81 ---225776.22 ms/batch
Step 00011200 L_out=0.129 Acc=0.86 ---249266.08 ms/batch
Step 00011250 L_out=0.100 Acc=0.83 ---273092.40 ms/batch
Step 00011300 L_out=0.106 Acc=0.87 ---296687.60 ms/batch
Round 2 | epoch=16, train costTime=315.2469229698181, | total_activation_sum=13036388
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.300 Acc=0.82 --- 5163.86 ms/batch
Step 00011400 L_out=0.109 Acc=0.86 ---28971.81 ms/batch
Step 00011450 L_out=0.155 Acc=0.88 ---52617.18 ms/batch
Step 00011500 L_out=0.235 Acc=0.68 ---76301.61 ms/batch
Step 00011550 L_out=0.129 Acc=0.81 ---99872.31 ms/batch
Step 00011600 L_out=0.217 Acc=0.73 ---123571.14 ms/batch
Step 00011650 L_out=0.273 Acc=0.87 ---147282.18 ms/batch
Step 00011700 L_out=0.108 Acc=0.80 ---170806.77 ms/batch
Step 00011750 L_out=0.189 Acc=0.81 ---194694.42 ms/batch
Step 00011800 L_out=0.124 Acc=0.86 ---218453.03 ms/batch
Step 00011850 L_out=0.216 Acc=0.84 ---242028.46 ms/batch
Step 00011900 L_out=0.281 Acc=0.87 ---265729.23 ms/batch
Step 00011950 L_out=0.169 Acc=0.82 ---289476.70 ms/batch
Step 00012000 L_out=0.165 Acc=0.84 ---312980.90 ms/batch
Round 2 | epoch=17, train costTime=315.7764263153076, | total_activation_sum=13005110
Round 2 | epoch=18, current m_iou=0.538112649496457
Round 2 | Best m_IoU is: 0.538, OA is: 0.876 | val costTime=118.60176515579224
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.146 Acc=0.82 ---20989.13 ms/batch
Step 00012100 L_out=0.169 Acc=0.88 ---44576.92 ms/batch
Step 00012150 L_out=0.290 Acc=0.81 ---68120.15 ms/batch
Step 00012200 L_out=0.242 Acc=0.82 ---91745.87 ms/batch
Step 00012250 L_out=0.098 Acc=0.89 ---115217.86 ms/batch
Step 00012300 L_out=0.030 Acc=0.83 ---138768.89 ms/batch
Step 00012350 L_out=0.146 Acc=0.88 ---162341.85 ms/batch
Step 00012400 L_out=0.342 Acc=0.81 ---185384.02 ms/batch
Step 00012450 L_out=0.061 Acc=0.88 ---208086.05 ms/batch
Step 00012500 L_out=0.207 Acc=0.85 ---231606.71 ms/batch
Step 00012550 L_out=0.221 Acc=0.79 ---254607.87 ms/batch
Step 00012600 L_out=0.081 Acc=0.90 ---278115.15 ms/batch
Step 00012650 L_out=0.238 Acc=0.91 ---301675.41 ms/batch
Round 2 | epoch=18, train costTime=345.94274520874023, | total_activation_sum=12989581
Round 2 | epoch=19, current m_iou=0.5609499147896106
Round 2 | Best m_IoU is: 0.561, OA is: 0.893 | val costTime=118.2682421207428
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.239 Acc=0.88 ---12726.49 ms/batch
Step 00012750 L_out=0.165 Acc=0.92 ---36288.04 ms/batch
Step 00012800 L_out=0.115 Acc=0.91 ---60078.99 ms/batch
Step 00012850 L_out=0.478 Acc=0.75 ---83412.18 ms/batch
Step 00012900 L_out=0.164 Acc=0.87 ---106985.30 ms/batch
Step 00012950 L_out=0.157 Acc=0.82 ---130456.13 ms/batch
Step 00013000 L_out=0.288 Acc=0.89 ---153928.04 ms/batch
Step 00013050 L_out=0.274 Acc=0.85 ---177470.69 ms/batch
Step 00013100 L_out=0.061 Acc=0.87 ---200957.07 ms/batch
Step 00013150 L_out=0.084 Acc=0.88 ---224340.19 ms/batch
Step 00013200 L_out=0.137 Acc=0.86 ---247906.16 ms/batch
Step 00013250 L_out=0.207 Acc=0.80 ---271431.74 ms/batch
Step 00013300 L_out=0.286 Acc=0.88 ---294956.69 ms/batch
Round 2 | epoch=19, train costTime=313.9768385887146, | total_activation_sum=12929899
Round 2 | epoch=20, current m_iou=0.5620686741460967
Round 2 | Best m_IoU is: 0.562, OA is: 0.888 | val costTime=118.17249345779419
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.071 Acc=0.86 --- 4747.57 ms/batch
Step 00013400 L_out=0.582 Acc=0.81 ---28271.66 ms/batch
Step 00013450 L_out=0.354 Acc=0.85 ---51899.95 ms/batch
Step 00013500 L_out=0.140 Acc=0.77 ---75596.74 ms/batch
Step 00013550 L_out=0.346 Acc=0.88 ---99138.09 ms/batch
Step 00013600 L_out=0.338 Acc=0.89 ---122826.58 ms/batch
Step 00013650 L_out=0.165 Acc=0.90 ---146313.73 ms/batch
Step 00013700 L_out=0.180 Acc=0.89 ---170240.99 ms/batch
Step 00013750 L_out=0.087 Acc=0.89 ---193824.77 ms/batch
Step 00013800 L_out=0.227 Acc=0.87 ---217458.01 ms/batch
Step 00013850 L_out=0.126 Acc=0.86 ---241120.85 ms/batch
Step 00013900 L_out=0.384 Acc=0.85 ---264775.71 ms/batch
Step 00013950 L_out=0.240 Acc=0.85 ---288356.34 ms/batch
Step 00014000 L_out=0.146 Acc=0.85 ---311870.21 ms/batch
Round 2 | epoch=20, train costTime=315.09654784202576, | total_activation_sum=12968148
Round 2 | epoch=21, current m_iou=0.5816232727058325
Round 2 | Best m_IoU is: 0.582, OA is: 0.895 | val costTime=117.61004757881165
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.161 Acc=0.84 ---20347.92 ms/batch
Step 00014100 L_out=0.345 Acc=0.80 ---43959.05 ms/batch
Step 00014150 L_out=0.085 Acc=0.94 ---67591.87 ms/batch
Step 00014200 L_out=0.072 Acc=0.84 ---91230.15 ms/batch
Step 00014250 L_out=0.129 Acc=0.91 ---114887.04 ms/batch
Step 00014300 L_out=0.165 Acc=0.75 ---138576.33 ms/batch
Step 00014350 L_out=0.099 Acc=0.84 ---162126.65 ms/batch
Step 00014400 L_out=0.040 Acc=0.90 ---185659.40 ms/batch
Step 00014450 L_out=0.188 Acc=0.85 ---209141.67 ms/batch
Step 00014500 L_out=0.334 Acc=0.87 ---232847.50 ms/batch
Step 00014550 L_out=0.342 Acc=0.86 ---256557.63 ms/batch
Step 00014600 L_out=0.182 Acc=0.88 ---280097.93 ms/batch
Step 00014650 L_out=0.178 Acc=0.81 ---303690.47 ms/batch
Round 2 | epoch=21, train costTime=314.9167854785919, | total_activation_sum=13089865
Round 2 | epoch=22, current m_iou=0.5886677896019202
Round 2 | Best m_IoU is: 0.589, OA is: 0.896 | val costTime=118.32363963127136
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.093 Acc=0.89 ---12351.62 ms/batch
Step 00014750 L_out=0.406 Acc=0.85 ---36004.04 ms/batch
Step 00014800 L_out=0.083 Acc=0.91 ---59618.21 ms/batch
Step 00014850 L_out=0.415 Acc=0.84 ---83196.33 ms/batch
Step 00014900 L_out=0.240 Acc=0.88 ---106758.26 ms/batch
Step 00014950 L_out=0.243 Acc=0.90 ---130366.02 ms/batch
Step 00015000 L_out=0.224 Acc=0.77 ---154016.70 ms/batch
Step 00015050 L_out=0.105 Acc=0.91 ---177511.88 ms/batch
Step 00015100 L_out=0.127 Acc=0.89 ---201083.53 ms/batch
Step 00015150 L_out=0.358 Acc=0.85 ---224753.69 ms/batch
Step 00015200 L_out=0.064 Acc=0.87 ---248453.16 ms/batch
Step 00015250 L_out=0.170 Acc=0.93 ---272182.63 ms/batch
Step 00015300 L_out=0.078 Acc=0.88 ---295743.22 ms/batch
Round 2 | epoch=22, train costTime=315.23909091949463, | total_activation_sum=12871596
Round 2 | epoch=23, current m_iou=0.5826015638542777
Round 2 | Best m_IoU is: 0.589, OA is: 0.896 | val costTime=117.7507917881012
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.213 Acc=0.87 --- 4319.98 ms/batch
Step 00015400 L_out=0.101 Acc=0.90 ---28279.50 ms/batch
Step 00015450 L_out=0.114 Acc=0.89 ---51863.68 ms/batch
Step 00015500 L_out=0.174 Acc=0.84 ---75684.85 ms/batch
Step 00015550 L_out=0.216 Acc=0.83 ---99253.91 ms/batch
Step 00015600 L_out=0.490 Acc=0.77 ---122787.45 ms/batch
Step 00015650 L_out=0.242 Acc=0.70 ---146379.57 ms/batch
Step 00015700 L_out=0.465 Acc=0.86 ---169840.32 ms/batch
Step 00015750 L_out=0.192 Acc=0.90 ---193589.69 ms/batch
Step 00015800 L_out=0.150 Acc=0.91 ---217351.48 ms/batch
Step 00015850 L_out=0.173 Acc=0.92 ---240947.04 ms/batch
Step 00015900 L_out=0.110 Acc=0.76 ---264526.68 ms/batch
Step 00015950 L_out=0.147 Acc=0.91 ---288124.56 ms/batch
Step 00016000 L_out=0.210 Acc=0.84 ---311732.24 ms/batch
Round 2 | epoch=23, train costTime=315.5326247215271, | total_activation_sum=12989954
Round 2 | epoch=24, current m_iou=0.5936114868181344
Round 2 | Best m_IoU is: 0.594, OA is: 0.897 | val costTime=117.90066528320312
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.131 Acc=0.89 ---19760.47 ms/batch
Step 00016100 L_out=0.137 Acc=0.88 ---43215.68 ms/batch
Step 00016150 L_out=0.114 Acc=0.90 ---66677.92 ms/batch
Step 00016200 L_out=0.188 Acc=0.90 ---90426.37 ms/batch
Step 00016250 L_out=0.296 Acc=0.84 ---113977.62 ms/batch
Step 00016300 L_out=0.229 Acc=0.84 ---137576.73 ms/batch
Step 00016350 L_out=0.134 Acc=0.89 ---161113.42 ms/batch
Step 00016400 L_out=0.285 Acc=0.87 ---184863.10 ms/batch
Step 00016450 L_out=0.297 Acc=0.88 ---208627.26 ms/batch
Step 00016500 L_out=0.200 Acc=0.95 ---232325.57 ms/batch
Step 00016550 L_out=0.067 Acc=0.92 ---256126.66 ms/batch
Step 00016600 L_out=0.121 Acc=0.87 ---279923.48 ms/batch
Step 00016650 L_out=0.225 Acc=0.81 ---303362.80 ms/batch
Round 2 | epoch=24, train costTime=315.22069358825684, | total_activation_sum=13008822
Round 2 | epoch=25, current m_iou=0.5958943589220613
Round 2 | Best m_IoU is: 0.596, OA is: 0.894 | val costTime=117.98673415184021
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.077 Acc=0.91 ---11864.30 ms/batch
Step 00016750 L_out=0.119 Acc=0.91 ---35524.46 ms/batch
Step 00016800 L_out=0.142 Acc=0.88 ---59139.53 ms/batch
Step 00016850 L_out=0.055 Acc=0.90 ---82750.87 ms/batch
Step 00016900 L_out=0.062 Acc=0.84 ---106638.63 ms/batch
Step 00016950 L_out=1.268 Acc=0.61 ---130018.90 ms/batch
Step 00017000 L_out=0.160 Acc=0.85 ---153797.22 ms/batch
Step 00017050 L_out=0.235 Acc=0.91 ---177368.90 ms/batch
Step 00017100 L_out=0.073 Acc=0.77 ---201193.51 ms/batch
Step 00017150 L_out=0.245 Acc=0.91 ---225007.83 ms/batch
Step 00017200 L_out=0.145 Acc=0.85 ---248520.10 ms/batch
Step 00017250 L_out=0.560 Acc=0.70 ---271938.83 ms/batch
Step 00017300 L_out=0.190 Acc=0.89 ---295683.99 ms/batch
Round 2 | epoch=25, train costTime=315.7303087711334, | total_activation_sum=12934535
Round 2 | epoch=26, current m_iou=0.5928027649479597
Round 2 | Best m_IoU is: 0.596, OA is: 0.894 | val costTime=117.66344380378723
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.134 Acc=0.81 --- 3872.73 ms/batch
Step 00017400 L_out=0.212 Acc=0.81 ---27314.09 ms/batch
Step 00017450 L_out=0.036 Acc=0.88 ---50962.90 ms/batch
Step 00017500 L_out=0.116 Acc=0.89 ---74444.22 ms/batch
Step 00017550 L_out=0.247 Acc=0.84 ---98145.79 ms/batch
Step 00017600 L_out=0.027 Acc=0.79 ---121763.59 ms/batch
Step 00017650 L_out=0.121 Acc=0.91 ---145383.63 ms/batch
Step 00017700 L_out=0.114 Acc=0.91 ---169134.91 ms/batch
Step 00017750 L_out=0.084 Acc=0.75 ---192653.96 ms/batch
Step 00017800 L_out=0.086 Acc=0.90 ---216284.26 ms/batch
Step 00017850 L_out=0.214 Acc=0.95 ---239944.27 ms/batch
Step 00017900 L_out=0.082 Acc=0.80 ---263675.58 ms/batch
Step 00017950 L_out=0.206 Acc=0.80 ---287368.97 ms/batch
Step 00018000 L_out=0.153 Acc=0.85 ---310953.18 ms/batch
Round 2 | epoch=26, train costTime=315.24637937545776, | total_activation_sum=13006792
Round 2 | epoch=27, current m_iou=0.5917442108356368
Round 2 | Best m_IoU is: 0.596, OA is: 0.894 | val costTime=117.39075970649719
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.347 Acc=0.86 ---19346.27 ms/batch
Step 00018100 L_out=0.315 Acc=0.84 ---43005.04 ms/batch
Step 00018150 L_out=0.210 Acc=0.87 ---66957.01 ms/batch
Step 00018200 L_out=0.102 Acc=0.80 ---90443.05 ms/batch
Step 00018250 L_out=0.155 Acc=0.88 ---114092.86 ms/batch
Step 00018300 L_out=0.035 Acc=0.90 ---137628.40 ms/batch
Step 00018350 L_out=0.160 Acc=0.79 ---161359.56 ms/batch
Step 00018400 L_out=0.153 Acc=0.87 ---185187.83 ms/batch
Step 00018450 L_out=0.104 Acc=0.91 ---208657.34 ms/batch
Step 00018500 L_out=0.164 Acc=0.86 ---232197.42 ms/batch
Step 00018550 L_out=0.044 Acc=0.92 ---255670.50 ms/batch
Step 00018600 L_out=0.125 Acc=0.88 ---279352.66 ms/batch
Step 00018650 L_out=0.119 Acc=0.90 ---303008.79 ms/batch
Round 2 | epoch=27, train costTime=315.118350982666, | total_activation_sum=13094376
Round 2 | epoch=28, current m_iou=0.5964177239121674
Round 2 | Best m_IoU is: 0.596, OA is: 0.891 | val costTime=117.24295234680176
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.230 Acc=0.75 ---11334.10 ms/batch
Step 00018750 L_out=0.179 Acc=0.78 ---34948.73 ms/batch
Step 00018800 L_out=0.354 Acc=0.85 ---58442.82 ms/batch
Step 00018850 L_out=0.087 Acc=0.91 ---82131.63 ms/batch
Step 00018900 L_out=0.276 Acc=0.86 ---105945.32 ms/batch
Step 00018950 L_out=0.131 Acc=0.92 ---129708.30 ms/batch
Step 00019000 L_out=0.094 Acc=0.94 ---153269.15 ms/batch
Step 00019050 L_out=0.538 Acc=0.77 ---176901.77 ms/batch
Step 00019100 L_out=0.127 Acc=0.87 ---200467.63 ms/batch
Step 00019150 L_out=0.314 Acc=0.83 ---224206.28 ms/batch
Step 00019200 L_out=0.140 Acc=0.86 ---247898.71 ms/batch
Step 00019250 L_out=0.082 Acc=0.81 ---271595.32 ms/batch
Step 00019300 L_out=0.239 Acc=0.82 ---295122.41 ms/batch
Round 2 | epoch=28, train costTime=315.4543676376343, | total_activation_sum=12982306
Round 2 | epoch=29, current m_iou=0.6015751394133151
Round 2 | Best m_IoU is: 0.602, OA is: 0.894 | val costTime=117.47544932365417
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.112 Acc=0.87 --- 3361.32 ms/batch
Step 00019400 L_out=0.332 Acc=0.93 ---27176.95 ms/batch
Step 00019450 L_out=0.164 Acc=0.86 ---50759.34 ms/batch
Step 00019500 L_out=0.301 Acc=0.88 ---74311.34 ms/batch
Step 00019550 L_out=0.091 Acc=0.89 ---97865.81 ms/batch
Step 00019600 L_out=0.115 Acc=0.89 ---121579.59 ms/batch
Step 00019650 L_out=0.166 Acc=0.89 ---145400.93 ms/batch
Step 00019700 L_out=0.280 Acc=0.85 ---168959.15 ms/batch
Step 00019750 L_out=0.277 Acc=0.82 ---192580.07 ms/batch
Step 00019800 L_out=0.189 Acc=0.86 ---216434.57 ms/batch
Step 00019850 L_out=0.120 Acc=0.89 ---240100.77 ms/batch
Step 00019900 L_out=0.300 Acc=0.77 ---263668.30 ms/batch
Step 00019950 L_out=0.185 Acc=0.88 ---287283.99 ms/batch
Step 00020000 L_out=0.103 Acc=0.87 ---310934.15 ms/batch
Round 2 | epoch=29, train costTime=315.5890600681305, | total_activation_sum=12895218
Round 2 | epoch=30, current m_iou=0.5978184840593876
Round 2 | Best m_IoU is: 0.602, OA is: 0.894 | val costTime=117.67902302742004
Round 2 | ****EPOCH 30****
Round 3 | ****EPOCH 0****
Step 00020050 L_out=1.322 Acc=0.66 ---18809.70 ms/batch
Step 00020100 L_out=0.805 Acc=0.63 ---42768.27 ms/batch
Step 00020150 L_out=0.520 Acc=0.55 ---66491.97 ms/batch
Step 00020200 L_out=1.009 Acc=0.77 ---90167.60 ms/batch
Step 00020250 L_out=0.611 Acc=0.81 ---113844.30 ms/batch
Step 00020300 L_out=1.349 Acc=0.35 ---137675.21 ms/batch
Step 00020350 L_out=0.420 Acc=0.81 ---161658.07 ms/batch
Step 00020400 L_out=0.864 Acc=0.81 ---185293.31 ms/batch
Step 00020450 L_out=0.525 Acc=0.80 ---209039.24 ms/batch
Step 00020500 L_out=0.721 Acc=0.63 ---232722.05 ms/batch
Step 00020550 L_out=0.594 Acc=0.74 ---256251.81 ms/batch
Step 00020600 L_out=0.288 Acc=0.88 ---279687.48 ms/batch
Step 00020650 L_out=0.897 Acc=0.83 ---303341.78 ms/batch
Round 3 | epoch=0, train costTime=316.103138923645, | total_activation_sum=22260159
Round 3 | ****EPOCH 1****
Step 00020700 L_out=0.411 Acc=0.92 ---10841.64 ms/batch
Step 00020750 L_out=0.651 Acc=0.80 ---34430.58 ms/batch
Step 00020800 L_out=0.365 Acc=0.90 ---58212.40 ms/batch
Step 00020850 L_out=0.587 Acc=0.83 ---81936.89 ms/batch
Step 00020900 L_out=0.590 Acc=0.76 ---105921.77 ms/batch
Step 00020950 L_out=0.727 Acc=0.82 ---129787.88 ms/batch
Step 00021000 L_out=0.544 Acc=0.70 ---153444.13 ms/batch
Step 00021050 L_out=0.457 Acc=0.79 ---176966.54 ms/batch
Step 00021100 L_out=0.669 Acc=0.76 ---200773.87 ms/batch
Step 00021150 L_out=0.547 Acc=0.81 ---224440.07 ms/batch
Step 00021200 L_out=0.378 Acc=0.86 ---248196.51 ms/batch
Step 00021250 L_out=0.354 Acc=0.81 ---271890.59 ms/batch
Step 00021300 L_out=0.856 Acc=0.84 ---295539.34 ms/batch
Round 3 | epoch=1, train costTime=316.5808379650116, | total_activation_sum=21124321
Round 3 | ****EPOCH 2****
Step 00021350 L_out=0.502 Acc=0.91 --- 2856.46 ms/batch
Step 00021400 L_out=0.763 Acc=0.75 ---26616.97 ms/batch
Step 00021450 L_out=0.241 Acc=0.73 ---50286.30 ms/batch
Step 00021500 L_out=1.075 Acc=0.81 ---74059.07 ms/batch
Step 00021550 L_out=1.741 Acc=0.80 ---97836.41 ms/batch
Step 00021600 L_out=1.149 Acc=0.89 ---121622.33 ms/batch
Step 00021650 L_out=0.307 Acc=0.87 ---145404.23 ms/batch
Step 00021700 L_out=0.485 Acc=0.77 ---169103.45 ms/batch
Step 00021750 L_out=0.465 Acc=0.85 ---192939.41 ms/batch
Step 00021800 L_out=0.692 Acc=0.68 ---216662.61 ms/batch
Step 00021850 L_out=0.818 Acc=0.78 ---240388.21 ms/batch
Step 00021900 L_out=0.353 Acc=0.89 ---264057.16 ms/batch
Step 00021950 L_out=0.644 Acc=0.79 ---287742.08 ms/batch
Step 00022000 L_out=0.583 Acc=0.84 ---311490.01 ms/batch
Round 3 | epoch=2, train costTime=316.72641348838806, | total_activation_sum=20859813
Round 3 | ****EPOCH 3****
Step 00022050 L_out=0.351 Acc=0.89 ---18477.81 ms/batch
Step 00022100 L_out=2.352 Acc=0.67 ---42392.65 ms/batch
Step 00022150 L_out=0.623 Acc=0.82 ---66134.52 ms/batch
Step 00022200 L_out=0.430 Acc=0.82 ---89691.92 ms/batch
Step 00022250 L_out=0.808 Acc=0.75 ---113434.37 ms/batch
Step 00022300 L_out=0.358 Acc=0.90 ---137061.21 ms/batch
Step 00022350 L_out=0.292 Acc=0.90 ---160703.10 ms/batch
Step 00022400 L_out=0.410 Acc=0.84 ---184468.51 ms/batch
Step 00022450 L_out=0.745 Acc=0.75 ---208248.43 ms/batch
Step 00022500 L_out=0.750 Acc=0.91 ---231689.51 ms/batch
Step 00022550 L_out=0.433 Acc=0.72 ---255407.29 ms/batch
Step 00022600 L_out=0.476 Acc=0.79 ---278930.69 ms/batch
Step 00022650 L_out=1.093 Acc=0.67 ---302551.30 ms/batch
Round 3 | epoch=3, train costTime=315.8528530597687, | total_activation_sum=21046570
Round 3 | ****EPOCH 4****
Step 00022700 L_out=0.708 Acc=0.64 ---10504.70 ms/batch
Step 00022750 L_out=0.251 Acc=0.92 ---34063.15 ms/batch
Step 00022800 L_out=0.776 Acc=0.81 ---57676.81 ms/batch
Step 00022850 L_out=0.360 Acc=0.83 ---81312.78 ms/batch
Step 00022900 L_out=0.325 Acc=0.87 ---105003.55 ms/batch
Step 00022950 L_out=0.571 Acc=0.64 ---128604.67 ms/batch
Step 00023000 L_out=0.701 Acc=0.72 ---152108.25 ms/batch
Step 00023050 L_out=0.314 Acc=0.82 ---176082.24 ms/batch
Step 00023100 L_out=0.511 Acc=0.85 ---199754.82 ms/batch
Step 00023150 L_out=0.489 Acc=0.74 ---223469.55 ms/batch
Step 00023200 L_out=0.257 Acc=0.82 ---247336.41 ms/batch
Step 00023250 L_out=0.400 Acc=0.82 ---271043.41 ms/batch
Step 00023300 L_out=0.660 Acc=0.84 ---294970.01 ms/batch
Round 3 | epoch=4, train costTime=316.3573508262634, | total_activation_sum=21096163
Round 3 | ****EPOCH 5****
Step 00023350 L_out=0.336 Acc=0.82 --- 2331.66 ms/batch
Step 00023400 L_out=0.436 Acc=0.85 ---25983.73 ms/batch
Step 00023450 L_out=0.801 Acc=0.84 ---49620.29 ms/batch
Step 00023500 L_out=0.726 Acc=0.92 ---73433.60 ms/batch
Step 00023550 L_out=0.355 Acc=0.91 ---97209.56 ms/batch
Step 00023600 L_out=0.424 Acc=0.89 ---121019.17 ms/batch
Step 00023650 L_out=0.388 Acc=0.87 ---144778.54 ms/batch
Step 00023700 L_out=0.814 Acc=0.91 ---168543.29 ms/batch
Step 00023750 L_out=1.075 Acc=0.83 ---192066.54 ms/batch
Step 00023800 L_out=0.447 Acc=0.86 ---215954.48 ms/batch
Step 00023850 L_out=0.308 Acc=0.90 ---239588.73 ms/batch
Step 00023900 L_out=1.101 Acc=0.70 ---263047.43 ms/batch
Step 00023950 L_out=0.875 Acc=0.89 ---286620.59 ms/batch
Step 00024000 L_out=0.411 Acc=0.83 ---310324.23 ms/batch
Round 3 | epoch=5, train costTime=316.10919737815857, | total_activation_sum=21010104
Round 3 | ****EPOCH 6****
Step 00024050 L_out=0.439 Acc=0.93 ---17905.50 ms/batch
Step 00024100 L_out=0.161 Acc=0.92 ---41397.33 ms/batch
Step 00024150 L_out=0.424 Acc=0.88 ---65020.21 ms/batch
Step 00024200 L_out=0.697 Acc=0.63 ---88569.34 ms/batch
Step 00024250 L_out=0.884 Acc=0.79 ---112340.98 ms/batch
Step 00024300 L_out=1.032 Acc=0.77 ---136249.50 ms/batch
Step 00024350 L_out=0.616 Acc=0.80 ---159876.89 ms/batch
Step 00024400 L_out=0.417 Acc=0.85 ---183352.22 ms/batch
Step 00024450 L_out=0.156 Acc=0.93 ---207093.92 ms/batch
Step 00024500 L_out=0.713 Acc=0.74 ---230749.64 ms/batch
Step 00024550 L_out=0.495 Acc=0.79 ---254286.74 ms/batch
Step 00024600 L_out=0.265 Acc=0.92 ---277916.89 ms/batch
Step 00024650 L_out=0.574 Acc=0.90 ---301671.24 ms/batch
Round 3 | epoch=6, train costTime=315.42286014556885, | total_activation_sum=21044389
Round 3 | ****EPOCH 7****
Step 00024700 L_out=0.447 Acc=0.84 --- 9920.13 ms/batch
Step 00024750 L_out=0.327 Acc=0.91 ---33393.68 ms/batch
Step 00024800 L_out=0.698 Acc=0.80 ---57211.50 ms/batch
Step 00024850 L_out=0.312 Acc=0.89 ---80933.28 ms/batch
Step 00024900 L_out=0.391 Acc=0.87 ---104687.48 ms/batch
Step 00024950 L_out=0.271 Acc=0.93 ---128135.60 ms/batch
Step 00025000 L_out=0.552 Acc=0.80 ---151909.31 ms/batch
Step 00025050 L_out=0.303 Acc=0.88 ---175664.24 ms/batch
Step 00025100 L_out=0.252 Acc=0.77 ---199248.73 ms/batch
Step 00025150 L_out=0.454 Acc=0.89 ---222900.27 ms/batch
Step 00025200 L_out=0.266 Acc=0.97 ---246369.07 ms/batch
Step 00025250 L_out=0.339 Acc=0.91 ---270245.53 ms/batch
Step 00025300 L_out=0.398 Acc=0.94 ---293837.60 ms/batch
Round 3 | epoch=7, train costTime=315.59920740127563, | total_activation_sum=20902245
Round 3 | ****EPOCH 8****
Step 00025350 L_out=0.220 Acc=0.95 --- 1821.29 ms/batch
Step 00025400 L_out=0.605 Acc=0.75 ---25729.95 ms/batch
Step 00025450 L_out=0.146 Acc=0.92 ---49434.11 ms/batch
Step 00025500 L_out=0.158 Acc=0.91 ---73229.07 ms/batch
Step 00025550 L_out=0.145 Acc=0.91 ---96879.57 ms/batch
Step 00025600 L_out=0.217 Acc=0.97 ---120441.05 ms/batch
Step 00025650 L_out=0.488 Acc=0.90 ---144020.97 ms/batch
Step 00025700 L_out=0.114 Acc=0.88 ---167604.98 ms/batch
Step 00025750 L_out=0.194 Acc=0.93 ---191309.43 ms/batch
Step 00025800 L_out=0.148 Acc=0.84 ---214965.01 ms/batch
Step 00025850 L_out=0.258 Acc=0.91 ---238852.19 ms/batch
Step 00025900 L_out=0.303 Acc=0.87 ---262795.52 ms/batch
Step 00025950 L_out=0.189 Acc=0.90 ---286593.25 ms/batch
Step 00026000 L_out=0.129 Acc=0.90 ---310134.13 ms/batch
Round 3 | epoch=8, train costTime=316.29773330688477, | total_activation_sum=21175690
Round 3 | ****EPOCH 9****
Step 00026050 L_out=0.343 Acc=0.79 ---17425.49 ms/batch
Step 00026100 L_out=0.428 Acc=0.89 ---41018.84 ms/batch
Step 00026150 L_out=0.373 Acc=0.93 ---64801.51 ms/batch
Step 00026200 L_out=0.442 Acc=0.85 ---88462.01 ms/batch
Step 00026250 L_out=0.375 Acc=0.91 ---112086.68 ms/batch
Step 00026300 L_out=0.461 Acc=0.89 ---135716.60 ms/batch
Step 00026350 L_out=0.384 Acc=0.88 ---159518.67 ms/batch
Step 00026400 L_out=0.200 Acc=0.95 ---183353.62 ms/batch
Step 00026450 L_out=0.213 Acc=0.93 ---207163.03 ms/batch
Step 00026500 L_out=0.217 Acc=0.86 ---230699.73 ms/batch
Step 00026550 L_out=0.319 Acc=0.88 ---254378.96 ms/batch
Step 00026600 L_out=0.191 Acc=0.87 ---278053.67 ms/batch
Step 00026650 L_out=0.266 Acc=0.87 ---301809.38 ms/batch
Round 3 | epoch=9, train costTime=315.9449746608734, | total_activation_sum=21023541
Round 3 | ****EPOCH 10****
Step 00026700 L_out=0.561 Acc=0.87 --- 9530.24 ms/batch
Step 00026750 L_out=0.682 Acc=0.86 ---33260.03 ms/batch
Step 00026800 L_out=0.143 Acc=0.96 ---57144.82 ms/batch
Step 00026850 L_out=0.258 Acc=0.83 ---80903.42 ms/batch
Step 00026900 L_out=0.357 Acc=0.82 ---104585.14 ms/batch
Step 00026950 L_out=0.478 Acc=0.88 ---128425.62 ms/batch
Step 00027000 L_out=0.592 Acc=0.72 ---152337.40 ms/batch
Step 00027050 L_out=0.234 Acc=0.93 ---176195.11 ms/batch
Step 00027100 L_out=0.800 Acc=0.92 ---200086.13 ms/batch
Step 00027150 L_out=0.426 Acc=0.89 ---223628.39 ms/batch
Step 00027200 L_out=0.129 Acc=0.90 ---247441.78 ms/batch
Step 00027250 L_out=0.291 Acc=0.93 ---271127.86 ms/batch
Step 00027300 L_out=0.272 Acc=0.75 ---294940.22 ms/batch
Round 3 | epoch=10, train costTime=317.2724597454071, | total_activation_sum=21092188
Round 3 | ****EPOCH 11****
Step 00027350 L_out=0.343 Acc=0.90 --- 1473.24 ms/batch
Step 00027400 L_out=0.305 Acc=0.91 ---25069.69 ms/batch
Step 00027450 L_out=0.288 Acc=0.91 ---48698.02 ms/batch
Step 00027500 L_out=0.134 Acc=0.92 ---72509.17 ms/batch
Step 00027550 L_out=0.306 Acc=0.86 ---96061.88 ms/batch
Step 00027600 L_out=0.333 Acc=0.91 ---119927.62 ms/batch
Step 00027650 L_out=0.643 Acc=0.83 ---143598.87 ms/batch
Step 00027700 L_out=0.702 Acc=0.92 ---167467.39 ms/batch
Step 00027750 L_out=0.598 Acc=0.90 ---191270.43 ms/batch
Step 00027800 L_out=0.481 Acc=0.79 ---214999.36 ms/batch
Step 00027850 L_out=0.789 Acc=0.79 ---238839.60 ms/batch
Step 00027900 L_out=0.494 Acc=0.82 ---262504.06 ms/batch
Step 00027950 L_out=0.213 Acc=0.92 ---286309.73 ms/batch
Step 00028000 L_out=0.403 Acc=0.80 ---309847.56 ms/batch
Round 3 | epoch=11, train costTime=316.4499337673187, | total_activation_sum=21086441
Round 3 | ****EPOCH 12****
Step 00028050 L_out=0.332 Acc=0.88 ---17094.89 ms/batch
Step 00028100 L_out=0.130 Acc=0.97 ---40768.92 ms/batch
Step 00028150 L_out=0.328 Acc=0.88 ---64510.15 ms/batch
Step 00028200 L_out=0.264 Acc=0.84 ---88235.11 ms/batch
Step 00028250 L_out=0.452 Acc=0.83 ---111779.00 ms/batch
Step 00028300 L_out=0.103 Acc=0.93 ---135365.77 ms/batch
Step 00028350 L_out=0.789 Acc=0.77 ---159139.12 ms/batch
Step 00028400 L_out=0.233 Acc=0.88 ---182848.50 ms/batch
Step 00028450 L_out=0.396 Acc=0.89 ---206549.85 ms/batch
Step 00028500 L_out=0.152 Acc=0.96 ---230210.59 ms/batch
Step 00028550 L_out=0.391 Acc=0.87 ---253841.13 ms/batch
Step 00028600 L_out=0.257 Acc=0.92 ---277550.10 ms/batch
Step 00028650 L_out=0.200 Acc=0.90 ---301319.15 ms/batch
Round 3 | epoch=12, train costTime=315.98484086990356, | total_activation_sum=21089822
Round 3 | ****EPOCH 13****
Step 00028700 L_out=0.285 Acc=0.90 --- 8956.48 ms/batch
Step 00028750 L_out=0.226 Acc=0.93 ---32658.97 ms/batch
Step 00028800 L_out=0.339 Acc=0.90 ---56365.72 ms/batch
Step 00028850 L_out=0.423 Acc=0.87 ---79797.30 ms/batch
Step 00028900 L_out=0.282 Acc=0.82 ---103793.92 ms/batch
Step 00028950 L_out=0.251 Acc=0.92 ---127478.68 ms/batch
Step 00029000 L_out=0.515 Acc=0.79 ---151249.70 ms/batch
Step 00029050 L_out=0.234 Acc=0.87 ---174762.93 ms/batch
Step 00029100 L_out=0.234 Acc=0.92 ---198280.62 ms/batch
Step 00029150 L_out=0.153 Acc=0.91 ---221904.23 ms/batch
Step 00029200 L_out=0.325 Acc=0.91 ---245460.09 ms/batch
Step 00029250 L_out=0.368 Acc=0.80 ---269167.00 ms/batch
Step 00029300 L_out=0.127 Acc=0.93 ---292999.94 ms/batch
Round 3 | epoch=13, train costTime=315.84706115722656, | total_activation_sum=20997039
Round 3 | ****EPOCH 14****
Step 00029350 L_out=0.257 Acc=0.91 ---  951.95 ms/batch
Step 00029400 L_out=0.484 Acc=0.81 ---24533.55 ms/batch
Step 00029450 L_out=0.289 Acc=0.94 ---48281.69 ms/batch
Step 00029500 L_out=0.231 Acc=0.84 ---72116.73 ms/batch
Step 00029550 L_out=0.136 Acc=0.95 ---95772.23 ms/batch
Step 00029600 L_out=0.327 Acc=0.85 ---119476.87 ms/batch
Step 00029650 L_out=0.366 Acc=0.87 ---143071.42 ms/batch
Step 00029700 L_out=0.156 Acc=0.94 ---166749.33 ms/batch
Step 00029750 L_out=0.159 Acc=0.97 ---190506.58 ms/batch
Step 00029800 L_out=0.214 Acc=0.94 ---214254.49 ms/batch
Step 00029850 L_out=0.333 Acc=0.91 ---238093.71 ms/batch
Step 00029900 L_out=0.396 Acc=0.86 ---261670.25 ms/batch
Step 00029950 L_out=0.567 Acc=0.88 ---285138.12 ms/batch
Step 00030000 L_out=0.215 Acc=0.89 ---308798.55 ms/batch
Round 3 | epoch=14, train costTime=315.957724571228, | total_activation_sum=20935390
Round 3 | ****EPOCH 15****
Step 00030050 L_out=0.358 Acc=0.94 ---16587.08 ms/batch
Step 00030100 L_out=0.370 Acc=0.94 ---40264.53 ms/batch
Step 00030150 L_out=0.454 Acc=0.84 ---63596.78 ms/batch
Step 00030200 L_out=0.127 Acc=0.91 ---87333.01 ms/batch
Step 00030250 L_out=0.205 Acc=0.94 ---110958.37 ms/batch
Step 00030300 L_out=0.420 Acc=0.92 ---134882.90 ms/batch
Step 00030350 L_out=0.161 Acc=0.87 ---158723.79 ms/batch
Step 00030400 L_out=0.198 Acc=0.92 ---182388.01 ms/batch
Step 00030450 L_out=0.348 Acc=0.89 ---206192.97 ms/batch
Step 00030500 L_out=0.248 Acc=0.84 ---229730.90 ms/batch
Step 00030550 L_out=0.094 Acc=0.92 ---253443.39 ms/batch
Step 00030600 L_out=0.269 Acc=0.90 ---277313.78 ms/batch
Step 00030650 L_out=0.161 Acc=0.91 ---301036.39 ms/batch
Round 3 | epoch=15, train costTime=316.2531769275665, | total_activation_sum=21042795
Round 3 | ****EPOCH 16****
Step 00030700 L_out=0.285 Acc=0.92 --- 8580.13 ms/batch
Step 00030750 L_out=0.729 Acc=0.79 ---32223.22 ms/batch
Step 00030800 L_out=0.481 Acc=0.89 ---56243.10 ms/batch
Step 00030850 L_out=0.152 Acc=0.91 ---80147.46 ms/batch
Step 00030900 L_out=0.229 Acc=0.96 ---103947.31 ms/batch
Step 00030950 L_out=0.312 Acc=0.88 ---127601.53 ms/batch
Step 00031000 L_out=0.969 Acc=0.85 ---151309.84 ms/batch
Step 00031050 L_out=0.221 Acc=0.91 ---175118.19 ms/batch
Step 00031100 L_out=0.283 Acc=0.94 ---198635.35 ms/batch
Step 00031150 L_out=0.305 Acc=0.88 ---222581.53 ms/batch
Step 00031200 L_out=0.192 Acc=0.91 ---246101.82 ms/batch
Step 00031250 L_out=0.287 Acc=0.92 ---269872.02 ms/batch
Step 00031300 L_out=0.216 Acc=0.93 ---293675.37 ms/batch
Round 3 | epoch=16, train costTime=317.1671051979065, | total_activation_sum=21031271
Round 3 | ****EPOCH 17****
Step 00031350 L_out=0.273 Acc=0.83 ---  440.13 ms/batch
Step 00031400 L_out=0.145 Acc=0.94 ---24312.55 ms/batch
Step 00031450 L_out=0.155 Acc=0.92 ---48186.62 ms/batch
Step 00031500 L_out=0.294 Acc=0.86 ---72060.78 ms/batch
Step 00031550 L_out=0.164 Acc=0.92 ---95946.66 ms/batch
Step 00031600 L_out=0.057 Acc=0.96 ---119738.96 ms/batch
Step 00031650 L_out=0.323 Acc=0.88 ---143521.55 ms/batch
Step 00031700 L_out=0.374 Acc=0.81 ---167080.24 ms/batch
Step 00031750 L_out=0.260 Acc=0.91 ---190757.81 ms/batch
Step 00031800 L_out=0.953 Acc=0.75 ---214517.76 ms/batch
Step 00031850 L_out=0.380 Acc=0.88 ---238259.16 ms/batch
Step 00031900 L_out=0.191 Acc=0.92 ---261932.57 ms/batch
Step 00031950 L_out=0.278 Acc=0.81 ---285723.80 ms/batch
Step 00032000 L_out=0.109 Acc=0.97 ---309425.70 ms/batch
Round 3 | epoch=17, train costTime=316.8642566204071, | total_activation_sum=21027392
Round 3 | epoch=18, current m_iou=0.676740406287248
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.71645474433899
Round 3 | ****EPOCH 18****
Step 00032050 L_out=0.283 Acc=0.94 ---16221.58 ms/batch
Step 00032100 L_out=0.177 Acc=0.96 ---39987.64 ms/batch
Step 00032150 L_out=0.241 Acc=0.87 ---63747.78 ms/batch
Step 00032200 L_out=0.255 Acc=0.87 ---87477.21 ms/batch
Step 00032250 L_out=0.233 Acc=0.93 ---111333.56 ms/batch
Step 00032300 L_out=0.327 Acc=0.85 ---135003.88 ms/batch
Step 00032350 L_out=0.190 Acc=0.89 ---158714.42 ms/batch
Step 00032400 L_out=0.159 Acc=0.95 ---182365.84 ms/batch
Step 00032450 L_out=0.401 Acc=0.79 ---206223.58 ms/batch
Step 00032500 L_out=0.260 Acc=0.74 ---229829.38 ms/batch
Step 00032550 L_out=0.231 Acc=0.90 ---253351.52 ms/batch
Step 00032600 L_out=0.163 Acc=0.96 ---276896.91 ms/batch
Step 00032650 L_out=0.248 Acc=0.87 ---300494.89 ms/batch
Round 3 | epoch=18, train costTime=316.23919773101807, | total_activation_sum=21131825
Round 3 | epoch=19, current m_iou=0.6631109926764723
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.49716854095459
Round 3 | ****EPOCH 19****
Step 00032700 L_out=0.062 Acc=0.96 --- 8061.84 ms/batch
Step 00032750 L_out=0.192 Acc=0.94 ---31972.97 ms/batch
Step 00032800 L_out=0.132 Acc=0.95 ---55738.65 ms/batch
Step 00032850 L_out=0.310 Acc=0.89 ---79487.32 ms/batch
Step 00032900 L_out=0.066 Acc=0.96 ---103252.57 ms/batch
Step 00032950 L_out=0.492 Acc=0.91 ---127074.20 ms/batch
Step 00033000 L_out=0.066 Acc=0.94 ---150875.12 ms/batch
Step 00033050 L_out=0.773 Acc=0.77 ---174542.98 ms/batch
Step 00033100 L_out=0.375 Acc=0.89 ---198394.41 ms/batch
Step 00033150 L_out=0.339 Acc=0.89 ---221947.53 ms/batch
Step 00033200 L_out=0.071 Acc=0.95 ---245641.99 ms/batch
Step 00033250 L_out=0.226 Acc=0.93 ---269287.43 ms/batch
Step 00033300 L_out=0.177 Acc=0.94 ---293014.59 ms/batch
Step 00033350 L_out=0.199 Acc=0.92 ---316729.72 ms/batch
Round 3 | epoch=19, train costTime=316.7301049232483, | total_activation_sum=21105200
Round 3 | epoch=20, current m_iou=0.6658027133846394
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.771151304245
Round 3 | ****EPOCH 20****
Step 00033400 L_out=0.223 Acc=0.94 ---23810.73 ms/batch
Step 00033450 L_out=0.193 Acc=0.93 ---47497.29 ms/batch
Step 00033500 L_out=0.192 Acc=0.93 ---71178.94 ms/batch
Step 00033550 L_out=0.214 Acc=0.92 ---95167.98 ms/batch
Step 00033600 L_out=0.078 Acc=0.94 ---118878.52 ms/batch
Step 00033650 L_out=0.743 Acc=0.87 ---142491.32 ms/batch
Step 00033700 L_out=0.190 Acc=0.92 ---166025.84 ms/batch
Step 00033750 L_out=0.475 Acc=0.86 ---189783.66 ms/batch
Step 00033800 L_out=0.214 Acc=0.92 ---213459.14 ms/batch
Step 00033850 L_out=0.121 Acc=0.92 ---237206.44 ms/batch
Step 00033900 L_out=0.390 Acc=0.84 ---260839.71 ms/batch
Step 00033950 L_out=0.244 Acc=0.90 ---284381.21 ms/batch
Step 00034000 L_out=0.149 Acc=0.92 ---308012.93 ms/batch
Round 3 | epoch=20, train costTime=316.0980520248413, | total_activation_sum=20969178
Round 3 | epoch=21, current m_iou=0.6680289900982841
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.55002045631409
Round 3 | ****EPOCH 21****
Step 00034050 L_out=0.327 Acc=0.92 ---15712.25 ms/batch
Step 00034100 L_out=0.225 Acc=0.83 ---39545.78 ms/batch
Step 00034150 L_out=0.326 Acc=0.89 ---63279.60 ms/batch
Step 00034200 L_out=0.128 Acc=0.82 ---86980.54 ms/batch
Step 00034250 L_out=0.608 Acc=0.85 ---110902.76 ms/batch
Step 00034300 L_out=0.358 Acc=0.92 ---134682.75 ms/batch
Step 00034350 L_out=0.475 Acc=0.88 ---158549.16 ms/batch
Step 00034400 L_out=0.574 Acc=0.92 ---181993.37 ms/batch
Step 00034450 L_out=0.174 Acc=0.95 ---205736.39 ms/batch
Step 00034500 L_out=0.166 Acc=0.89 ---229446.52 ms/batch
Step 00034550 L_out=0.142 Acc=0.94 ---253188.64 ms/batch
Step 00034600 L_out=0.276 Acc=0.94 ---276895.40 ms/batch
Step 00034650 L_out=0.195 Acc=0.86 ---300451.23 ms/batch
Round 3 | epoch=21, train costTime=316.60387992858887, | total_activation_sum=20964285
Round 3 | epoch=22, current m_iou=0.6699408247960267
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.31217193603516
Round 3 | ****EPOCH 22****
Step 00034700 L_out=0.619 Acc=0.86 --- 7585.51 ms/batch
Step 00034750 L_out=0.334 Acc=0.94 ---31172.58 ms/batch
Step 00034800 L_out=0.113 Acc=0.91 ---54956.51 ms/batch
Step 00034850 L_out=0.210 Acc=0.88 ---78623.72 ms/batch
Step 00034900 L_out=0.203 Acc=0.92 ---102282.70 ms/batch
Step 00034950 L_out=0.269 Acc=0.93 ---125896.97 ms/batch
Step 00035000 L_out=0.190 Acc=0.90 ---149548.32 ms/batch
Step 00035050 L_out=0.075 Acc=0.96 ---173148.05 ms/batch
Step 00035100 L_out=0.238 Acc=0.89 ---196930.44 ms/batch
Step 00035150 L_out=0.096 Acc=0.95 ---220639.90 ms/batch
Step 00035200 L_out=0.139 Acc=0.86 ---244626.89 ms/batch
Step 00035250 L_out=0.144 Acc=0.93 ---268310.44 ms/batch
Step 00035300 L_out=0.111 Acc=0.95 ---292017.26 ms/batch
Step 00035350 L_out=0.175 Acc=0.92 ---315804.90 ms/batch
Round 3 | epoch=22, train costTime=316.2751531600952, | total_activation_sum=21147238
Round 3 | epoch=23, current m_iou=0.6690451589390827
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.49133157730103
Round 3 | ****EPOCH 23****
Step 00035400 L_out=0.389 Acc=0.83 ---23340.28 ms/batch
Step 00035450 L_out=0.134 Acc=0.93 ---47081.38 ms/batch
Step 00035500 L_out=0.216 Acc=0.86 ---70864.91 ms/batch
Step 00035550 L_out=0.101 Acc=0.94 ---94328.79 ms/batch
Step 00035600 L_out=0.159 Acc=0.96 ---118056.26 ms/batch
Step 00035650 L_out=0.233 Acc=0.88 ---141855.35 ms/batch
Step 00035700 L_out=0.182 Acc=0.94 ---165330.93 ms/batch
Step 00035750 L_out=0.443 Acc=0.86 ---189192.26 ms/batch
Step 00035800 L_out=0.187 Acc=0.94 ---212719.41 ms/batch
Step 00035850 L_out=0.087 Acc=0.95 ---236371.25 ms/batch
Step 00035900 L_out=0.641 Acc=0.86 ---260096.57 ms/batch
Step 00035950 L_out=0.184 Acc=0.95 ---283657.81 ms/batch
Step 00036000 L_out=0.390 Acc=0.81 ---307195.68 ms/batch
Round 3 | epoch=23, train costTime=315.8191554546356, | total_activation_sum=21054119
Round 3 | epoch=24, current m_iou=0.6705816677010077
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.52862501144409
Round 3 | ****EPOCH 24****
Step 00036050 L_out=0.130 Acc=0.94 ---15085.60 ms/batch
Step 00036100 L_out=0.192 Acc=0.90 ---38737.46 ms/batch
Step 00036150 L_out=0.182 Acc=0.92 ---62270.88 ms/batch
Step 00036200 L_out=0.219 Acc=0.90 ---85718.52 ms/batch
Step 00036250 L_out=0.307 Acc=0.90 ---109532.05 ms/batch
Step 00036300 L_out=0.196 Acc=0.92 ---133275.07 ms/batch
Step 00036350 L_out=0.168 Acc=0.90 ---156688.98 ms/batch
Step 00036400 L_out=0.255 Acc=0.94 ---180317.45 ms/batch
Step 00036450 L_out=0.219 Acc=0.88 ---204023.51 ms/batch
Step 00036500 L_out=0.315 Acc=0.94 ---227589.09 ms/batch
Step 00036550 L_out=0.154 Acc=0.93 ---251199.85 ms/batch
Step 00036600 L_out=0.106 Acc=0.94 ---274732.45 ms/batch
Step 00036650 L_out=0.227 Acc=0.94 ---298288.67 ms/batch
Round 3 | epoch=24, train costTime=314.91658878326416, | total_activation_sum=21031945
Round 3 | epoch=25, current m_iou=0.6732951361656224
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.55806183815002
Round 3 | ****EPOCH 25****
Step 00036700 L_out=0.258 Acc=0.91 --- 6981.50 ms/batch
Step 00036750 L_out=0.248 Acc=0.93 ---30565.48 ms/batch
Step 00036800 L_out=0.302 Acc=0.92 ---54160.57 ms/batch
Step 00036850 L_out=0.086 Acc=0.95 ---77759.48 ms/batch
Step 00036900 L_out=0.095 Acc=0.96 ---101620.21 ms/batch
Step 00036950 L_out=0.537 Acc=0.75 ---125432.88 ms/batch
Step 00037000 L_out=0.264 Acc=0.95 ---149163.27 ms/batch
Step 00037050 L_out=0.277 Acc=0.89 ---173021.65 ms/batch
Step 00037100 L_out=0.106 Acc=0.95 ---196665.69 ms/batch
Step 00037150 L_out=0.227 Acc=0.94 ---220409.92 ms/batch
Step 00037200 L_out=0.247 Acc=0.92 ---244271.32 ms/batch
Step 00037250 L_out=0.152 Acc=0.94 ---267929.62 ms/batch
Step 00037300 L_out=0.207 Acc=0.91 ---291736.66 ms/batch
Step 00037350 L_out=0.416 Acc=0.87 ---315116.76 ms/batch
Round 3 | epoch=25, train costTime=316.0298373699188, | total_activation_sum=21156362
Round 3 | epoch=26, current m_iou=0.6701033141962969
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.31625843048096
Round 3 | ****EPOCH 26****
Step 00037400 L_out=0.169 Acc=0.94 ---22673.53 ms/batch
Step 00037450 L_out=0.148 Acc=0.96 ---46206.19 ms/batch
Step 00037500 L_out=0.260 Acc=0.89 ---69734.04 ms/batch
Step 00037550 L_out=0.110 Acc=0.92 ---93504.52 ms/batch
Step 00037600 L_out=0.127 Acc=0.95 ---117099.18 ms/batch
Step 00037650 L_out=0.143 Acc=0.94 ---140694.16 ms/batch
Step 00037700 L_out=0.243 Acc=0.92 ---164420.63 ms/batch
Step 00037750 L_out=0.260 Acc=0.89 ---188151.45 ms/batch
Step 00037800 L_out=0.113 Acc=0.94 ---211677.67 ms/batch
Step 00037850 L_out=0.090 Acc=0.97 ---235341.82 ms/batch
Step 00037900 L_out=0.404 Acc=0.89 ---259149.49 ms/batch
Step 00037950 L_out=0.294 Acc=0.87 ---282613.31 ms/batch
Step 00038000 L_out=0.288 Acc=0.89 ---306363.34 ms/batch
Round 3 | epoch=26, train costTime=315.2985053062439, | total_activation_sum=20864246
Round 3 | epoch=27, current m_iou=0.6759514344466517
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.60938358306885
Round 3 | ****EPOCH 27****
Step 00038050 L_out=0.370 Acc=0.85 ---14775.11 ms/batch
Step 00038100 L_out=0.176 Acc=0.94 ---38203.73 ms/batch
Step 00038150 L_out=0.220 Acc=0.89 ---61854.23 ms/batch
Step 00038200 L_out=0.476 Acc=0.86 ---85506.86 ms/batch
Step 00038250 L_out=0.438 Acc=0.90 ---109357.44 ms/batch
Step 00038300 L_out=0.128 Acc=0.94 ---132996.63 ms/batch
Step 00038350 L_out=0.178 Acc=0.95 ---156771.12 ms/batch
Step 00038400 L_out=0.245 Acc=0.89 ---180412.98 ms/batch
Step 00038450 L_out=0.286 Acc=0.83 ---203962.27 ms/batch
Step 00038500 L_out=0.276 Acc=0.92 ---227778.02 ms/batch
Step 00038550 L_out=0.169 Acc=0.94 ---251397.53 ms/batch
Step 00038600 L_out=0.778 Acc=0.74 ---275230.73 ms/batch
Step 00038650 L_out=0.110 Acc=0.93 ---298979.91 ms/batch
Round 3 | epoch=27, train costTime=315.8473880290985, | total_activation_sum=20945373
Round 3 | epoch=28, current m_iou=0.6757038618496192
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.60530734062195
Round 3 | ****EPOCH 28****
Step 00038700 L_out=0.096 Acc=0.95 --- 6608.10 ms/batch
Step 00038750 L_out=0.225 Acc=0.86 ---30543.83 ms/batch
Step 00038800 L_out=0.095 Acc=0.96 ---54215.88 ms/batch
Step 00038850 L_out=0.321 Acc=0.87 ---77992.83 ms/batch
Step 00038900 L_out=0.160 Acc=0.87 ---101844.73 ms/batch
Step 00038950 L_out=0.118 Acc=0.87 ---125502.10 ms/batch
Step 00039000 L_out=0.248 Acc=0.85 ---149241.89 ms/batch
Step 00039050 L_out=0.191 Acc=0.94 ---172947.38 ms/batch
Step 00039100 L_out=0.107 Acc=0.94 ---196590.25 ms/batch
Step 00039150 L_out=0.160 Acc=0.95 ---220275.09 ms/batch
Step 00039200 L_out=0.602 Acc=0.89 ---243874.07 ms/batch
Step 00039250 L_out=0.168 Acc=0.91 ---267953.83 ms/batch
Step 00039300 L_out=0.097 Acc=0.93 ---291666.14 ms/batch
Step 00039350 L_out=0.220 Acc=0.92 ---315131.83 ms/batch
Round 3 | epoch=28, train costTime=316.58128333091736, | total_activation_sum=21195626
Round 3 | epoch=29, current m_iou=0.6744226229846955
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.48072576522827
Round 3 | ****EPOCH 29****
Step 00039400 L_out=0.156 Acc=0.90 ---22337.21 ms/batch
Step 00039450 L_out=0.176 Acc=0.93 ---46113.08 ms/batch
Step 00039500 L_out=0.216 Acc=0.91 ---69608.57 ms/batch
Step 00039550 L_out=0.199 Acc=0.93 ---93335.74 ms/batch
Step 00039600 L_out=0.302 Acc=0.90 ---117082.25 ms/batch
Step 00039650 L_out=0.358 Acc=0.97 ---140949.81 ms/batch
Step 00039700 L_out=0.134 Acc=0.95 ---164306.67 ms/batch
Step 00039750 L_out=0.136 Acc=0.94 ---187879.99 ms/batch
Step 00039800 L_out=0.173 Acc=0.94 ---211298.24 ms/batch
Step 00039850 L_out=0.363 Acc=0.90 ---235069.46 ms/batch
Step 00039900 L_out=0.126 Acc=0.91 ---258657.58 ms/batch
Step 00039950 L_out=0.085 Acc=0.97 ---282353.82 ms/batch
Step 00040000 L_out=0.100 Acc=0.94 ---306236.05 ms/batch
Round 3 | epoch=29, train costTime=315.59407210350037, | total_activation_sum=21012892
Round 3 | epoch=30, current m_iou=0.6747694085990962
Round 3 | Best m_IoU is: 0.677, OA is: 0.915 | val costTime=118.56430768966675
Round 3 | ****EPOCH 30****
Round 4 | ****EPOCH 0****
Step 00040050 L_out=0.562 Acc=0.86 ---14210.33 ms/batch
Step 00040100 L_out=0.967 Acc=0.58 ---37867.24 ms/batch
Step 00040150 L_out=2.571 Acc=0.54 ---61601.40 ms/batch
Step 00040200 L_out=0.618 Acc=0.84 ---85199.28 ms/batch
Step 00040250 L_out=1.245 Acc=0.77 ---108861.24 ms/batch
Step 00040300 L_out=0.732 Acc=0.74 ---132464.15 ms/batch
Step 00040350 L_out=0.628 Acc=0.87 ---156290.52 ms/batch
Step 00040400 L_out=0.442 Acc=0.89 ---180189.14 ms/batch
Step 00040450 L_out=0.334 Acc=0.93 ---203960.12 ms/batch
Step 00040500 L_out=0.551 Acc=0.91 ---227984.45 ms/batch
Step 00040550 L_out=0.580 Acc=0.87 ---251652.17 ms/batch
Step 00040600 L_out=0.354 Acc=0.91 ---275439.80 ms/batch
Step 00040650 L_out=0.416 Acc=0.92 ---299066.37 ms/batch
Round 4 | epoch=0, train costTime=316.9057593345642, | total_activation_sum=27638499
Round 4 | ****EPOCH 1****
Step 00040700 L_out=0.822 Acc=0.85 --- 6027.71 ms/batch
Step 00040750 L_out=0.248 Acc=0.94 ---29663.07 ms/batch
Step 00040800 L_out=0.738 Acc=0.81 ---53516.43 ms/batch
Step 00040850 L_out=0.187 Acc=0.96 ---77463.02 ms/batch
Step 00040900 L_out=0.401 Acc=0.85 ---101154.41 ms/batch
Step 00040950 L_out=0.302 Acc=0.92 ---124690.21 ms/batch
Step 00041000 L_out=0.654 Acc=0.86 ---148319.35 ms/batch
Step 00041050 L_out=0.690 Acc=0.84 ---172074.73 ms/batch
Step 00041100 L_out=1.137 Acc=0.78 ---195860.50 ms/batch
Step 00041150 L_out=0.584 Acc=0.81 ---219754.47 ms/batch
Step 00041200 L_out=0.775 Acc=0.75 ---243324.04 ms/batch
Step 00041250 L_out=0.753 Acc=0.81 ---267077.62 ms/batch
Step 00041300 L_out=0.466 Acc=0.88 ---290621.83 ms/batch
Step 00041350 L_out=0.408 Acc=0.89 ---314284.27 ms/batch
Round 4 | epoch=1, train costTime=316.14787101745605, | total_activation_sum=26311926
Round 4 | ****EPOCH 2****
Step 00041400 L_out=0.239 Acc=0.94 ---21800.81 ms/batch
Step 00041450 L_out=0.398 Acc=0.87 ---45339.71 ms/batch
Step 00041500 L_out=0.234 Acc=0.92 ---68962.70 ms/batch
Step 00041550 L_out=0.190 Acc=0.93 ---92045.01 ms/batch
Step 00041600 L_out=0.303 Acc=0.93 ---115583.05 ms/batch
Step 00041650 L_out=0.319 Acc=0.93 ---139425.68 ms/batch
Step 00041700 L_out=0.830 Acc=0.82 ---163077.84 ms/batch
Step 00041750 L_out=1.811 Acc=0.72 ---186755.96 ms/batch
Step 00041800 L_out=0.725 Acc=0.78 ---210396.40 ms/batch
Step 00041850 L_out=0.508 Acc=0.88 ---234261.16 ms/batch
Step 00041900 L_out=2.041 Acc=0.71 ---257627.04 ms/batch
Step 00041950 L_out=0.444 Acc=0.84 ---281276.33 ms/batch
Step 00042000 L_out=0.324 Acc=0.91 ---304943.32 ms/batch
Round 4 | epoch=2, train costTime=314.77979135513306, | total_activation_sum=26428766
Round 4 | ****EPOCH 3****
Step 00042050 L_out=0.304 Acc=0.91 ---13705.29 ms/batch
Step 00042100 L_out=0.301 Acc=0.93 ---37369.77 ms/batch
Step 00042150 L_out=0.709 Acc=0.88 ---60981.28 ms/batch
Step 00042200 L_out=0.362 Acc=0.88 ---84386.40 ms/batch
Step 00042250 L_out=0.482 Acc=0.92 ---108057.37 ms/batch
Step 00042300 L_out=0.734 Acc=0.84 ---131768.98 ms/batch
Step 00042350 L_out=1.395 Acc=0.77 ---155515.76 ms/batch
Step 00042400 L_out=0.217 Acc=0.96 ---179039.53 ms/batch
Step 00042450 L_out=0.268 Acc=0.91 ---202647.03 ms/batch
Step 00042500 L_out=0.289 Acc=0.91 ---226428.96 ms/batch
Step 00042550 L_out=0.379 Acc=0.93 ---249827.23 ms/batch
Step 00042600 L_out=0.784 Acc=0.80 ---273751.48 ms/batch
Step 00042650 L_out=0.113 Acc=0.93 ---297473.80 ms/batch
Round 4 | epoch=3, train costTime=315.4403645992279, | total_activation_sum=26253830
Round 4 | ****EPOCH 4****
Step 00042700 L_out=0.794 Acc=0.82 --- 5668.81 ms/batch
Step 00042750 L_out=1.360 Acc=0.68 ---28615.98 ms/batch
Step 00042800 L_out=0.332 Acc=0.87 ---50501.80 ms/batch
Step 00042850 L_out=0.673 Acc=0.81 ---72400.44 ms/batch
Step 00042900 L_out=0.408 Acc=0.83 ---94606.00 ms/batch
Step 00042950 L_out=0.443 Acc=0.87 ---116602.32 ms/batch
Step 00043000 L_out=1.181 Acc=0.80 ---138942.35 ms/batch
Step 00043050 L_out=0.185 Acc=0.92 ---161188.33 ms/batch
Step 00043100 L_out=0.983 Acc=0.77 ---183226.96 ms/batch
Step 00043150 L_out=0.433 Acc=0.91 ---205398.79 ms/batch
Step 00043200 L_out=0.380 Acc=0.93 ---227489.89 ms/batch
Step 00043250 L_out=0.324 Acc=0.90 ---249720.14 ms/batch
Step 00043300 L_out=1.267 Acc=0.82 ---272450.15 ms/batch
Step 00043350 L_out=0.810 Acc=0.85 ---295859.00 ms/batch
Round 4 | epoch=4, train costTime=298.19082713127136, | total_activation_sum=26103868
Round 4 | ****EPOCH 5****
Step 00043400 L_out=0.462 Acc=0.94 ---21166.34 ms/batch
Step 00043450 L_out=2.917 Acc=0.78 ---44913.40 ms/batch
Step 00043500 L_out=0.622 Acc=0.86 ---68253.99 ms/batch
Step 00043550 L_out=0.330 Acc=0.94 ---92026.96 ms/batch
Step 00043600 L_out=0.457 Acc=0.90 ---115414.43 ms/batch
Step 00043650 L_out=0.363 Acc=0.87 ---139057.41 ms/batch
Step 00043700 L_out=0.916 Acc=0.88 ---162916.00 ms/batch
Step 00043750 L_out=0.354 Acc=0.91 ---186405.48 ms/batch
Step 00043800 L_out=0.191 Acc=0.95 ---210023.58 ms/batch
Step 00043850 L_out=0.914 Acc=0.73 ---233756.16 ms/batch
Step 00043900 L_out=0.663 Acc=0.86 ---257475.51 ms/batch
Step 00043950 L_out=0.717 Acc=0.85 ---280845.27 ms/batch
Step 00044000 L_out=1.604 Acc=0.58 ---304378.76 ms/batch
Round 4 | epoch=5, train costTime=314.8547372817993, | total_activation_sum=26423644
Round 4 | ****EPOCH 6****
Step 00044050 L_out=0.667 Acc=0.88 ---13311.47 ms/batch
Step 00044100 L_out=0.331 Acc=0.91 ---37009.73 ms/batch
Step 00044150 L_out=0.183 Acc=0.94 ---60610.36 ms/batch
Step 00044200 L_out=0.320 Acc=0.91 ---84279.57 ms/batch
Step 00044250 L_out=0.262 Acc=0.94 ---107794.65 ms/batch
Step 00044300 L_out=0.260 Acc=0.92 ---131532.45 ms/batch
Step 00044350 L_out=0.780 Acc=0.88 ---155289.85 ms/batch
Step 00044400 L_out=0.183 Acc=0.86 ---178707.77 ms/batch
Step 00044450 L_out=0.178 Acc=0.97 ---202409.92 ms/batch
Step 00044500 L_out=0.574 Acc=0.91 ---226050.52 ms/batch
Step 00044550 L_out=0.474 Acc=0.85 ---248736.01 ms/batch
Step 00044600 L_out=1.060 Acc=0.86 ---270480.74 ms/batch
Step 00044650 L_out=0.452 Acc=0.88 ---292311.89 ms/batch
Round 4 | epoch=6, train costTime=310.3509781360626, | total_activation_sum=26430860
Round 4 | ****EPOCH 7****
Step 00044700 L_out=0.417 Acc=0.88 --- 5275.70 ms/batch
Step 00044750 L_out=0.382 Acc=0.94 ---28841.52 ms/batch
Step 00044800 L_out=0.189 Acc=0.95 ---52473.61 ms/batch
Step 00044850 L_out=0.361 Acc=0.89 ---76346.08 ms/batch
Step 00044900 L_out=0.320 Acc=0.93 ---100088.64 ms/batch
Step 00044950 L_out=0.248 Acc=0.92 ---123852.85 ms/batch
Step 00045000 L_out=0.405 Acc=0.89 ---147561.20 ms/batch
Step 00045050 L_out=0.872 Acc=0.92 ---171160.82 ms/batch
Step 00045100 L_out=0.397 Acc=0.91 ---194898.21 ms/batch
Step 00045150 L_out=0.454 Acc=0.91 ---218642.89 ms/batch
Step 00045200 L_out=0.532 Acc=0.88 ---242325.40 ms/batch
Step 00045250 L_out=0.417 Acc=0.92 ---265999.42 ms/batch
Step 00045300 L_out=0.650 Acc=0.86 ---289405.66 ms/batch
Step 00045350 L_out=0.207 Acc=0.93 ---313019.78 ms/batch
Round 4 | epoch=7, train costTime=315.8899233341217, | total_activation_sum=26238836
Round 4 | ****EPOCH 8****
Step 00045400 L_out=0.701 Acc=0.84 ---20840.38 ms/batch
Step 00045450 L_out=0.256 Acc=0.92 ---44688.03 ms/batch
Step 00045500 L_out=0.372 Acc=0.95 ---68261.78 ms/batch
Step 00045550 L_out=0.507 Acc=0.92 ---92018.72 ms/batch
Step 00045600 L_out=0.292 Acc=0.90 ---115758.70 ms/batch
Step 00045650 L_out=0.425 Acc=0.89 ---139443.20 ms/batch
Step 00045700 L_out=0.527 Acc=0.93 ---163158.48 ms/batch
Step 00045750 L_out=0.232 Acc=0.92 ---186837.75 ms/batch
Step 00045800 L_out=0.252 Acc=0.96 ---210657.62 ms/batch
Step 00045850 L_out=0.329 Acc=0.91 ---234136.97 ms/batch
Step 00045900 L_out=0.229 Acc=0.91 ---257679.15 ms/batch
Step 00045950 L_out=0.192 Acc=0.93 ---281381.41 ms/batch
Step 00046000 L_out=0.275 Acc=0.88 ---305154.54 ms/batch
Round 4 | epoch=8, train costTime=316.0307478904724, | total_activation_sum=26364335
Round 4 | ****EPOCH 9****
Step 00046050 L_out=0.615 Acc=0.84 ---12593.82 ms/batch
Step 00046100 L_out=0.128 Acc=0.96 ---36425.39 ms/batch
Step 00046150 L_out=0.171 Acc=0.96 ---60038.29 ms/batch
Step 00046200 L_out=0.455 Acc=0.90 ---83732.02 ms/batch
Step 00046250 L_out=0.301 Acc=0.90 ---107382.45 ms/batch
Step 00046300 L_out=0.282 Acc=0.93 ---130863.51 ms/batch
Step 00046350 L_out=0.248 Acc=0.93 ---154354.94 ms/batch
Step 00046400 L_out=0.514 Acc=0.87 ---178090.55 ms/batch
Step 00046450 L_out=0.138 Acc=0.94 ---201698.31 ms/batch
Step 00046500 L_out=0.389 Acc=0.89 ---225434.71 ms/batch
Step 00046550 L_out=0.296 Acc=0.87 ---249091.99 ms/batch
Step 00046600 L_out=0.530 Acc=0.86 ---272796.68 ms/batch
Step 00046650 L_out=0.170 Acc=0.96 ---296481.62 ms/batch
Round 4 | epoch=9, train costTime=315.58048272132874, | total_activation_sum=26291442
Round 4 | ****EPOCH 10****
Step 00046700 L_out=0.328 Acc=0.87 --- 4861.01 ms/batch
Step 00046750 L_out=0.405 Acc=0.91 ---28498.97 ms/batch
Step 00046800 L_out=0.182 Acc=0.92 ---51895.58 ms/batch
Step 00046850 L_out=0.257 Acc=0.93 ---75835.50 ms/batch
Step 00046900 L_out=0.340 Acc=0.90 ---99530.29 ms/batch
Step 00046950 L_out=0.595 Acc=0.90 ---123190.63 ms/batch
Step 00047000 L_out=0.282 Acc=0.89 ---146944.02 ms/batch
Step 00047050 L_out=0.640 Acc=0.87 ---170827.01 ms/batch
Step 00047100 L_out=0.252 Acc=0.96 ---194475.43 ms/batch
Step 00047150 L_out=0.360 Acc=0.90 ---218063.49 ms/batch
Step 00047200 L_out=0.323 Acc=0.90 ---241738.96 ms/batch
Step 00047250 L_out=1.026 Acc=0.74 ---265505.94 ms/batch
Step 00047300 L_out=0.248 Acc=0.96 ---289313.83 ms/batch
Step 00047350 L_out=0.484 Acc=0.96 ---312945.32 ms/batch
Round 4 | epoch=10, train costTime=316.26863527297974, | total_activation_sum=26299415
Round 4 | ****EPOCH 11****
Step 00047400 L_out=0.276 Acc=0.91 ---20292.45 ms/batch
Step 00047450 L_out=0.135 Acc=0.94 ---44044.38 ms/batch
Step 00047500 L_out=0.574 Acc=0.92 ---67867.74 ms/batch
Step 00047550 L_out=0.766 Acc=0.84 ---91390.10 ms/batch
Step 00047600 L_out=0.497 Acc=0.92 ---114985.89 ms/batch
Step 00047650 L_out=0.268 Acc=0.91 ---138782.15 ms/batch
Step 00047700 L_out=0.309 Acc=0.94 ---162471.75 ms/batch
Step 00047750 L_out=0.494 Acc=0.90 ---186418.10 ms/batch
Step 00047800 L_out=0.800 Acc=0.81 ---210224.35 ms/batch
Step 00047850 L_out=0.509 Acc=0.93 ---233955.26 ms/batch
Step 00047900 L_out=0.364 Acc=0.96 ---257797.15 ms/batch
Step 00047950 L_out=0.355 Acc=0.91 ---281370.30 ms/batch
Step 00048000 L_out=0.192 Acc=0.96 ---304994.81 ms/batch
Round 4 | epoch=11, train costTime=316.31835293769836, | total_activation_sum=26286987
Round 4 | ****EPOCH 12****
Step 00048050 L_out=0.235 Acc=0.95 ---12312.05 ms/batch
Step 00048100 L_out=0.343 Acc=0.93 ---35997.30 ms/batch
Step 00048150 L_out=0.091 Acc=0.97 ---59463.84 ms/batch
Step 00048200 L_out=0.156 Acc=0.96 ---83056.70 ms/batch
Step 00048250 L_out=0.177 Acc=0.96 ---106720.35 ms/batch
Step 00048300 L_out=0.249 Acc=0.94 ---130314.90 ms/batch
Step 00048350 L_out=0.467 Acc=0.90 ---154100.84 ms/batch
Step 00048400 L_out=0.289 Acc=0.95 ---177859.77 ms/batch
Step 00048450 L_out=0.199 Acc=0.92 ---201376.86 ms/batch
Step 00048500 L_out=0.228 Acc=0.94 ---225115.18 ms/batch
Step 00048550 L_out=0.204 Acc=0.94 ---248774.87 ms/batch
Step 00048600 L_out=0.191 Acc=0.95 ---272464.83 ms/batch
Step 00048650 L_out=0.640 Acc=0.72 ---296101.12 ms/batch
Round 4 | epoch=12, train costTime=315.4852809906006, | total_activation_sum=26473475
Round 4 | ****EPOCH 13****
Step 00048700 L_out=0.859 Acc=0.68 --- 4240.43 ms/batch
Step 00048750 L_out=0.189 Acc=0.94 ---27840.36 ms/batch
Step 00048800 L_out=0.382 Acc=0.95 ---51727.37 ms/batch
Step 00048850 L_out=0.489 Acc=0.79 ---75559.35 ms/batch
Step 00048900 L_out=0.650 Acc=0.79 ---99184.24 ms/batch
Step 00048950 L_out=0.365 Acc=0.92 ---122738.08 ms/batch
Step 00049000 L_out=0.252 Acc=0.94 ---146218.25 ms/batch
Step 00049050 L_out=0.454 Acc=0.90 ---169910.81 ms/batch
Step 00049100 L_out=0.161 Acc=0.94 ---193401.82 ms/batch
Step 00049150 L_out=0.152 Acc=0.97 ---217241.63 ms/batch
Step 00049200 L_out=0.251 Acc=0.94 ---241161.53 ms/batch
Step 00049250 L_out=0.155 Acc=0.92 ---264752.56 ms/batch
Step 00049300 L_out=0.204 Acc=0.95 ---288756.94 ms/batch
Step 00049350 L_out=0.216 Acc=0.96 ---312509.23 ms/batch
Round 4 | epoch=13, train costTime=316.2634017467499, | total_activation_sum=26198937
Round 4 | ****EPOCH 14****
Step 00049400 L_out=0.363 Acc=0.92 ---19788.94 ms/batch
Step 00049450 L_out=0.491 Acc=0.85 ---43371.23 ms/batch
Step 00049500 L_out=0.370 Acc=0.91 ---67219.81 ms/batch
Step 00049550 L_out=0.208 Acc=0.94 ---90942.69 ms/batch
Step 00049600 L_out=0.465 Acc=0.93 ---114604.84 ms/batch
Step 00049650 L_out=0.439 Acc=0.91 ---138408.90 ms/batch
Step 00049700 L_out=0.249 Acc=0.94 ---162064.54 ms/batch
Step 00049750 L_out=0.694 Acc=0.92 ---185775.63 ms/batch
Step 00049800 L_out=0.207 Acc=0.89 ---209598.00 ms/batch
Step 00049850 L_out=0.536 Acc=0.88 ---233472.51 ms/batch
Step 00049900 L_out=0.213 Acc=0.89 ---257128.75 ms/batch
Step 00049950 L_out=0.349 Acc=0.92 ---280898.74 ms/batch
Step 00050000 L_out=0.173 Acc=0.96 ---304481.13 ms/batch
Round 4 | epoch=14, train costTime=316.2943983078003, | total_activation_sum=26299316
Round 4 | ****EPOCH 15****
Step 00050050 L_out=0.185 Acc=0.95 ---11771.91 ms/batch
Step 00050100 L_out=0.440 Acc=0.89 ---35440.09 ms/batch
Step 00050150 L_out=0.181 Acc=0.93 ---59164.49 ms/batch
Step 00050200 L_out=0.180 Acc=0.95 ---82816.88 ms/batch
Step 00050250 L_out=0.221 Acc=0.86 ---106420.99 ms/batch
Step 00050300 L_out=1.127 Acc=0.87 ---130082.37 ms/batch
Step 00050350 L_out=0.348 Acc=0.90 ---153788.13 ms/batch
Step 00050400 L_out=0.670 Acc=0.83 ---177528.91 ms/batch
Step 00050450 L_out=0.227 Acc=0.94 ---201328.45 ms/batch
Step 00050500 L_out=0.262 Acc=0.92 ---224901.25 ms/batch
Step 00050550 L_out=0.430 Acc=0.88 ---248529.77 ms/batch
Step 00050600 L_out=0.423 Acc=0.85 ---272202.91 ms/batch
Step 00050650 L_out=1.031 Acc=0.90 ---295921.40 ms/batch
Round 4 | epoch=15, train costTime=315.82341384887695, | total_activation_sum=26301675
Round 4 | ****EPOCH 16****
Step 00050700 L_out=0.238 Acc=0.95 --- 3831.45 ms/batch
Step 00050750 L_out=0.452 Acc=0.90 ---27328.37 ms/batch
Step 00050800 L_out=0.439 Acc=0.93 ---50886.88 ms/batch
Step 00050850 L_out=0.131 Acc=0.97 ---74778.35 ms/batch
Step 00050900 L_out=0.189 Acc=0.97 ---98486.00 ms/batch
Step 00050950 L_out=0.275 Acc=0.95 ---122188.28 ms/batch
Step 00051000 L_out=0.144 Acc=0.93 ---145737.18 ms/batch
Step 00051050 L_out=0.044 Acc=0.96 ---168885.12 ms/batch
Step 00051100 L_out=0.320 Acc=0.90 ---190721.24 ms/batch
Step 00051150 L_out=0.325 Acc=0.94 ---213269.18 ms/batch
Step 00051200 L_out=0.272 Acc=0.95 ---235903.94 ms/batch
Step 00051250 L_out=0.146 Acc=0.94 ---258008.98 ms/batch
Step 00051300 L_out=0.340 Acc=0.89 ---280498.67 ms/batch
Step 00051350 L_out=0.446 Acc=0.94 ---303349.43 ms/batch
Round 4 | epoch=16, train costTime=307.42346358299255, | total_activation_sum=26455479
Round 4 | ****EPOCH 17****
Step 00051400 L_out=0.318 Acc=0.93 ---18320.68 ms/batch
Step 00051450 L_out=0.303 Acc=0.95 ---41599.95 ms/batch
Step 00051500 L_out=0.168 Acc=0.93 ---63957.85 ms/batch
Step 00051550 L_out=0.873 Acc=0.93 ---87723.69 ms/batch
Step 00051600 L_out=0.364 Acc=0.90 ---111478.21 ms/batch
Step 00051650 L_out=0.285 Acc=0.92 ---135158.67 ms/batch
Step 00051700 L_out=0.171 Acc=0.95 ---159035.74 ms/batch
Step 00051750 L_out=0.098 Acc=0.94 ---182623.98 ms/batch
Step 00051800 L_out=0.236 Acc=0.90 ---206279.62 ms/batch
Step 00051850 L_out=0.369 Acc=0.92 ---230152.03 ms/batch
Step 00051900 L_out=0.494 Acc=0.94 ---253799.52 ms/batch
Step 00051950 L_out=0.110 Acc=0.95 ---277534.70 ms/batch
Step 00052000 L_out=0.340 Acc=0.88 ---301193.74 ms/batch
Round 4 | epoch=17, train costTime=313.3874583244324, | total_activation_sum=26381163
Round 4 | epoch=18, current m_iou=0.6560834388588654
Round 4 | Best m_IoU is: 0.656, OA is: 0.898 | val costTime=119.05176997184753
Round 4 | ****EPOCH 18****
Step 00052050 L_out=0.212 Acc=0.96 ---11368.58 ms/batch
Step 00052100 L_out=0.195 Acc=0.93 ---35019.78 ms/batch
Step 00052150 L_out=0.286 Acc=0.87 ---58907.38 ms/batch
Step 00052200 L_out=0.191 Acc=0.89 ---82630.75 ms/batch
Step 00052250 L_out=0.107 Acc=0.96 ---106332.67 ms/batch
Step 00052300 L_out=0.118 Acc=0.95 ---129787.18 ms/batch
Step 00052350 L_out=0.249 Acc=0.91 ---153570.28 ms/batch
Step 00052400 L_out=0.195 Acc=0.93 ---176926.93 ms/batch
Step 00052450 L_out=0.239 Acc=0.93 ---200514.99 ms/batch
Step 00052500 L_out=0.280 Acc=0.92 ---224137.93 ms/batch
Step 00052550 L_out=0.133 Acc=0.95 ---248024.14 ms/batch
Step 00052600 L_out=0.239 Acc=0.86 ---271647.36 ms/batch
Step 00052650 L_out=0.557 Acc=0.89 ---295161.00 ms/batch
Round 4 | epoch=18, train costTime=315.3890759944916, | total_activation_sum=26133852
Round 4 | epoch=19, current m_iou=0.6810517236475769
Round 4 | Best m_IoU is: 0.681, OA is: 0.913 | val costTime=118.69242572784424
Round 4 | ****EPOCH 19****
Step 00052700 L_out=0.502 Acc=0.78 --- 3390.68 ms/batch
Step 00052750 L_out=0.585 Acc=0.91 ---27096.17 ms/batch
Step 00052800 L_out=0.274 Acc=0.93 ---50890.89 ms/batch
Step 00052850 L_out=0.032 Acc=0.99 ---74416.74 ms/batch
Step 00052900 L_out=0.239 Acc=0.94 ---98048.07 ms/batch
Step 00052950 L_out=0.085 Acc=0.98 ---122060.35 ms/batch
Step 00053000 L_out=0.203 Acc=0.95 ---145635.94 ms/batch
Step 00053050 L_out=0.064 Acc=0.96 ---169353.58 ms/batch
Step 00053100 L_out=0.196 Acc=0.93 ---192840.52 ms/batch
Step 00053150 L_out=0.327 Acc=0.92 ---216474.12 ms/batch
Step 00053200 L_out=0.228 Acc=0.94 ---240084.02 ms/batch
Step 00053250 L_out=0.167 Acc=0.95 ---263925.35 ms/batch
Step 00053300 L_out=0.346 Acc=0.91 ---287542.40 ms/batch
Step 00053350 L_out=0.403 Acc=0.88 ---311404.78 ms/batch
Round 4 | epoch=19, train costTime=316.08538722991943, | total_activation_sum=26427729
Round 4 | epoch=20, current m_iou=0.6915180714163932
Round 4 | Best m_IoU is: 0.692, OA is: 0.917 | val costTime=118.76445841789246
Round 4 | ****EPOCH 20****
Step 00053400 L_out=0.190 Acc=0.93 ---19140.85 ms/batch
Step 00053450 L_out=0.387 Acc=0.90 ---42926.48 ms/batch
Step 00053500 L_out=0.457 Acc=0.93 ---66525.02 ms/batch
Step 00053550 L_out=0.230 Acc=0.94 ---90200.17 ms/batch
Step 00053600 L_out=0.271 Acc=0.94 ---113921.99 ms/batch
Step 00053650 L_out=0.507 Acc=0.95 ---137517.25 ms/batch
Step 00053700 L_out=0.321 Acc=0.89 ---161133.44 ms/batch
Step 00053750 L_out=0.095 Acc=0.97 ---184752.33 ms/batch
Step 00053800 L_out=0.373 Acc=0.92 ---208348.57 ms/batch
Step 00053850 L_out=0.350 Acc=0.92 ---231985.68 ms/batch
Step 00053900 L_out=0.251 Acc=0.92 ---255681.14 ms/batch
Step 00053950 L_out=0.399 Acc=0.90 ---279265.57 ms/batch
Step 00054000 L_out=0.584 Acc=0.85 ---302876.11 ms/batch
Round 4 | epoch=20, train costTime=315.5409038066864, | total_activation_sum=26246887
Round 4 | epoch=21, current m_iou=0.6853842256477882
Round 4 | Best m_IoU is: 0.692, OA is: 0.917 | val costTime=118.68003582954407
Round 4 | ****EPOCH 21****
Step 00054050 L_out=0.182 Acc=0.96 ---10944.74 ms/batch
Step 00054100 L_out=0.086 Acc=0.97 ---34709.50 ms/batch
Step 00054150 L_out=0.203 Acc=0.94 ---58422.83 ms/batch
Step 00054200 L_out=0.298 Acc=0.96 ---82311.86 ms/batch
Step 00054250 L_out=0.099 Acc=0.97 ---105802.15 ms/batch
Step 00054300 L_out=0.087 Acc=0.96 ---129533.10 ms/batch
Step 00054350 L_out=0.205 Acc=0.97 ---153133.95 ms/batch
Step 00054400 L_out=0.331 Acc=0.90 ---176893.12 ms/batch
Step 00054450 L_out=0.140 Acc=0.95 ---200586.71 ms/batch
Step 00054500 L_out=0.135 Acc=0.95 ---224295.24 ms/batch
Step 00054550 L_out=0.122 Acc=0.95 ---247946.26 ms/batch
Step 00054600 L_out=0.211 Acc=0.92 ---271387.44 ms/batch
Step 00054650 L_out=0.212 Acc=0.92 ---294991.22 ms/batch
Round 4 | epoch=21, train costTime=315.8584372997284, | total_activation_sum=26295831
Round 4 | epoch=22, current m_iou=0.6865830681043217
Round 4 | Best m_IoU is: 0.692, OA is: 0.917 | val costTime=118.75948524475098
Round 4 | ****EPOCH 22****
Step 00054700 L_out=0.208 Acc=0.92 --- 2807.93 ms/batch
Step 00054750 L_out=0.333 Acc=0.94 ---26536.22 ms/batch
Step 00054800 L_out=0.119 Acc=0.96 ---50277.93 ms/batch
Step 00054850 L_out=0.089 Acc=0.95 ---73955.45 ms/batch
Step 00054900 L_out=0.166 Acc=0.94 ---97669.95 ms/batch
Step 00054950 L_out=0.155 Acc=0.93 ---121382.89 ms/batch
Step 00055000 L_out=0.272 Acc=0.91 ---145081.60 ms/batch
Step 00055050 L_out=0.178 Acc=0.97 ---169223.56 ms/batch
Step 00055100 L_out=1.030 Acc=0.87 ---192878.39 ms/batch
Step 00055150 L_out=0.127 Acc=0.96 ---216579.98 ms/batch
Step 00055200 L_out=0.189 Acc=0.97 ---240322.18 ms/batch
Step 00055250 L_out=0.204 Acc=0.93 ---264047.69 ms/batch
Step 00055300 L_out=0.295 Acc=0.96 ---287664.37 ms/batch
Step 00055350 L_out=0.179 Acc=0.96 ---311359.61 ms/batch
Round 4 | epoch=22, train costTime=316.4841978549957, | total_activation_sum=26504424
Round 4 | epoch=23, current m_iou=0.6896568186600924
Round 4 | Best m_IoU is: 0.692, OA is: 0.917 | val costTime=118.4735050201416
Round 4 | ****EPOCH 23****
Step 00055400 L_out=0.177 Acc=0.95 ---18308.63 ms/batch
Step 00055450 L_out=0.212 Acc=0.94 ---41797.14 ms/batch
Step 00055500 L_out=0.270 Acc=0.91 ---65403.76 ms/batch
Step 00055550 L_out=0.310 Acc=0.92 ---88905.83 ms/batch
Step 00055600 L_out=0.221 Acc=0.95 ---112519.43 ms/batch
Step 00055650 L_out=0.306 Acc=0.95 ---136043.51 ms/batch
Step 00055700 L_out=0.274 Acc=0.85 ---159770.26 ms/batch
Step 00055750 L_out=0.080 Acc=0.97 ---183409.63 ms/batch
Step 00055800 L_out=0.090 Acc=0.97 ---207039.82 ms/batch
Step 00055850 L_out=0.121 Acc=0.96 ---230645.90 ms/batch
Step 00055900 L_out=0.396 Acc=0.94 ---254325.31 ms/batch
Step 00055950 L_out=0.171 Acc=0.96 ---278080.36 ms/batch
Step 00056000 L_out=0.073 Acc=0.98 ---301700.39 ms/batch
Round 4 | epoch=23, train costTime=315.1128168106079, | total_activation_sum=26361052
Round 4 | epoch=24, current m_iou=0.6947187769275767
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=119.17152309417725
Round 4 | ****EPOCH 24****
Step 00056050 L_out=0.126 Acc=0.92 ---10537.15 ms/batch
Step 00056100 L_out=0.076 Acc=0.96 ---34161.33 ms/batch
Step 00056150 L_out=0.336 Acc=0.94 ---57990.15 ms/batch
Step 00056200 L_out=0.214 Acc=0.95 ---81562.11 ms/batch
Step 00056250 L_out=0.116 Acc=0.97 ---105061.85 ms/batch
Step 00056300 L_out=0.246 Acc=0.91 ---128855.51 ms/batch
Step 00056350 L_out=0.180 Acc=0.94 ---152660.88 ms/batch
Step 00056400 L_out=0.200 Acc=0.93 ---176292.70 ms/batch
Step 00056450 L_out=0.278 Acc=0.94 ---199666.64 ms/batch
Step 00056500 L_out=0.217 Acc=0.93 ---223277.52 ms/batch
Step 00056550 L_out=0.141 Acc=0.92 ---246781.63 ms/batch
Step 00056600 L_out=0.317 Acc=0.90 ---270320.22 ms/batch
Step 00056650 L_out=0.168 Acc=0.94 ---293915.86 ms/batch
Round 4 | epoch=24, train costTime=315.081449508667, | total_activation_sum=26146260
Round 4 | epoch=25, current m_iou=0.6899815345007344
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=118.60746622085571
Round 4 | ****EPOCH 25****
Step 00056700 L_out=0.326 Acc=0.96 --- 2394.88 ms/batch
Step 00056750 L_out=0.189 Acc=0.94 ---26062.19 ms/batch
Step 00056800 L_out=0.209 Acc=0.93 ---49498.53 ms/batch
Step 00056850 L_out=0.175 Acc=0.97 ---73102.76 ms/batch
Step 00056900 L_out=0.142 Acc=0.94 ---96777.82 ms/batch
Step 00056950 L_out=0.125 Acc=0.97 ---120398.11 ms/batch
Step 00057000 L_out=0.264 Acc=0.90 ---144052.97 ms/batch
Step 00057050 L_out=0.599 Acc=0.93 ---167751.61 ms/batch
Step 00057100 L_out=0.177 Acc=0.97 ---191413.05 ms/batch
Step 00057150 L_out=0.184 Acc=0.95 ---215138.19 ms/batch
Step 00057200 L_out=0.110 Acc=0.95 ---238863.34 ms/batch
Step 00057250 L_out=0.256 Acc=0.96 ---262386.24 ms/batch
Step 00057300 L_out=0.830 Acc=0.82 ---286244.67 ms/batch
Step 00057350 L_out=0.162 Acc=0.92 ---309932.94 ms/batch
Round 4 | epoch=25, train costTime=315.61676239967346, | total_activation_sum=26276530
Round 4 | epoch=26, current m_iou=0.689311871706906
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=118.33197283744812
Round 4 | ****EPOCH 26****
Step 00057400 L_out=0.177 Acc=0.97 ---18170.78 ms/batch
Step 00057450 L_out=0.184 Acc=0.94 ---41895.23 ms/batch
Step 00057500 L_out=0.213 Acc=0.95 ---65708.27 ms/batch
Step 00057550 L_out=0.408 Acc=0.96 ---89400.49 ms/batch
Step 00057600 L_out=0.176 Acc=0.96 ---113046.55 ms/batch
Step 00057650 L_out=0.133 Acc=0.91 ---136705.96 ms/batch
Step 00057700 L_out=0.244 Acc=0.98 ---160178.54 ms/batch
Step 00057750 L_out=0.272 Acc=0.92 ---183884.35 ms/batch
Step 00057800 L_out=0.407 Acc=0.88 ---207597.62 ms/batch
Step 00057850 L_out=0.253 Acc=0.92 ---231296.08 ms/batch
Step 00057900 L_out=0.122 Acc=0.96 ---254988.73 ms/batch
Step 00057950 L_out=0.176 Acc=0.95 ---278627.21 ms/batch
Step 00058000 L_out=0.174 Acc=0.95 ---302393.13 ms/batch
Round 4 | epoch=26, train costTime=316.26837706565857, | total_activation_sum=26298269
Round 4 | epoch=27, current m_iou=0.6925599716243624
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=118.08792638778687
Round 4 | ****EPOCH 27****
Step 00058050 L_out=0.207 Acc=0.96 ---10195.83 ms/batch
Step 00058100 L_out=0.192 Acc=0.96 ---33771.50 ms/batch
Step 00058150 L_out=0.227 Acc=0.95 ---57250.25 ms/batch
Step 00058200 L_out=0.333 Acc=0.95 ---80794.34 ms/batch
Step 00058250 L_out=0.271 Acc=0.89 ---104506.01 ms/batch
Step 00058300 L_out=0.306 Acc=0.93 ---128203.05 ms/batch
Step 00058350 L_out=0.137 Acc=0.96 ---151732.39 ms/batch
Step 00058400 L_out=0.291 Acc=0.90 ---175138.04 ms/batch
Step 00058450 L_out=0.159 Acc=0.89 ---198529.25 ms/batch
Step 00058500 L_out=0.183 Acc=0.95 ---221989.89 ms/batch
Step 00058550 L_out=0.082 Acc=0.98 ---245519.20 ms/batch
Step 00058600 L_out=0.239 Acc=0.95 ---269055.97 ms/batch
Step 00058650 L_out=0.118 Acc=0.97 ---292685.57 ms/batch
Round 4 | epoch=27, train costTime=314.38578510284424, | total_activation_sum=26263848
Round 4 | epoch=28, current m_iou=0.6880197895214517
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=123.77637553215027
Round 4 | ****EPOCH 28****
Step 00058700 L_out=0.189 Acc=0.97 --- 1794.14 ms/batch
Step 00058750 L_out=0.136 Acc=0.97 ---25174.93 ms/batch
Step 00058800 L_out=0.130 Acc=0.96 ---48621.61 ms/batch
Step 00058850 L_out=0.332 Acc=0.87 ---92319.18 ms/batch
Step 00058900 L_out=0.254 Acc=0.93 ---116219.77 ms/batch
Step 00058950 L_out=0.157 Acc=0.92 ---139683.78 ms/batch
Step 00059000 L_out=0.285 Acc=0.93 ---163378.67 ms/batch
Step 00059050 L_out=0.093 Acc=0.97 ---186938.20 ms/batch
Step 00059100 L_out=0.109 Acc=0.97 ---210614.78 ms/batch
Step 00059150 L_out=0.102 Acc=0.97 ---234279.04 ms/batch
Step 00059200 L_out=0.138 Acc=0.93 ---257871.16 ms/batch
Step 00059250 L_out=0.112 Acc=0.96 ---281541.90 ms/batch
Step 00059300 L_out=0.262 Acc=0.86 ---305246.07 ms/batch
Step 00059350 L_out=0.114 Acc=0.97 ---328920.85 ms/batch
Round 4 | epoch=28, train costTime=335.0679757595062, | total_activation_sum=26320926
Round 4 | epoch=29, current m_iou=0.6865852601433374
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=139.13447666168213
Round 4 | ****EPOCH 29****
Step 00059400 L_out=0.117 Acc=0.98 ---17502.30 ms/batch
Step 00059450 L_out=0.210 Acc=0.95 ---41067.49 ms/batch
Step 00059500 L_out=0.131 Acc=0.96 ---64441.98 ms/batch
Step 00059550 L_out=0.261 Acc=0.95 ---88161.83 ms/batch
Step 00059600 L_out=0.183 Acc=0.96 ---112037.15 ms/batch
Step 00059650 L_out=0.467 Acc=0.89 ---135744.62 ms/batch
Step 00059700 L_out=0.251 Acc=0.93 ---159407.99 ms/batch
Step 00059750 L_out=0.051 Acc=0.98 ---183011.28 ms/batch
Step 00059800 L_out=0.220 Acc=0.95 ---206725.76 ms/batch
Step 00059850 L_out=0.261 Acc=0.93 ---230292.23 ms/batch
Step 00059900 L_out=0.358 Acc=0.93 ---254004.12 ms/batch
Step 00059950 L_out=0.203 Acc=0.95 ---277900.37 ms/batch
Step 00060000 L_out=0.282 Acc=0.93 ---301617.80 ms/batch
Round 4 | epoch=29, train costTime=315.59104919433594, | total_activation_sum=26336822
Round 4 | epoch=30, current m_iou=0.6897454479229531
Round 4 | Best m_IoU is: 0.695, OA is: 0.921 | val costTime=118.45507311820984
Round 4 | ****EPOCH 30****
Round 5 | ****EPOCH 0****
Step 00060050 L_out=2.814 Acc=0.60 --- 9438.51 ms/batch
Step 00060100 L_out=0.965 Acc=0.75 ---32961.42 ms/batch
Step 00060150 L_out=1.325 Acc=0.67 ---56580.14 ms/batch
Step 00060200 L_out=2.598 Acc=0.68 ---80177.93 ms/batch
Step 00060250 L_out=1.019 Acc=0.88 ---103847.80 ms/batch
Step 00060300 L_out=0.688 Acc=0.90 ---127861.52 ms/batch
Step 00060350 L_out=1.037 Acc=0.73 ---151508.99 ms/batch
Step 00060400 L_out=1.096 Acc=0.77 ---175273.01 ms/batch
Step 00060450 L_out=1.257 Acc=0.83 ---199184.26 ms/batch
Step 00060500 L_out=1.296 Acc=0.80 ---223078.92 ms/batch
Step 00060550 L_out=0.377 Acc=0.82 ---246958.29 ms/batch
Step 00060600 L_out=0.477 Acc=0.89 ---270664.65 ms/batch
Step 00060650 L_out=0.610 Acc=0.93 ---294403.78 ms/batch
Round 5 | epoch=0, train costTime=316.6876015663147, | total_activation_sum=32050745
Round 5 | ****EPOCH 1****
Step 00060700 L_out=0.565 Acc=0.89 --- 1435.77 ms/batch
Step 00060750 L_out=1.033 Acc=0.80 ---24837.68 ms/batch
Step 00060800 L_out=0.353 Acc=0.92 ---48795.62 ms/batch
Step 00060850 L_out=0.210 Acc=0.94 ---72627.41 ms/batch
Step 00060900 L_out=0.459 Acc=0.91 ---96543.36 ms/batch
Step 00060950 L_out=0.754 Acc=0.91 ---120318.27 ms/batch
Step 00061000 L_out=0.873 Acc=0.90 ---143905.86 ms/batch
Step 00061050 L_out=0.706 Acc=0.80 ---167646.65 ms/batch
Step 00061100 L_out=0.176 Acc=0.95 ---191372.37 ms/batch
Step 00061150 L_out=0.399 Acc=0.89 ---215051.85 ms/batch
Step 00061200 L_out=1.456 Acc=0.86 ---238876.09 ms/batch
Step 00061250 L_out=0.225 Acc=0.95 ---262718.67 ms/batch
Step 00061300 L_out=0.430 Acc=0.94 ---286388.46 ms/batch
Step 00061350 L_out=0.392 Acc=0.89 ---310119.41 ms/batch
Round 5 | epoch=1, train costTime=316.77829694747925, | total_activation_sum=30289401
Round 5 | ****EPOCH 2****
Step 00061400 L_out=0.608 Acc=0.84 ---16950.31 ms/batch
Step 00061450 L_out=0.846 Acc=0.91 ---40947.28 ms/batch
Step 00061500 L_out=0.216 Acc=0.96 ---64757.12 ms/batch
Step 00061550 L_out=0.398 Acc=0.94 ---88603.24 ms/batch
Step 00061600 L_out=0.207 Acc=0.82 ---112275.70 ms/batch
Step 00061650 L_out=0.545 Acc=0.92 ---135952.53 ms/batch
Step 00061700 L_out=0.357 Acc=0.89 ---159821.69 ms/batch
Step 00061750 L_out=0.569 Acc=0.90 ---183506.09 ms/batch
Step 00061800 L_out=0.285 Acc=0.91 ---207257.75 ms/batch
Step 00061850 L_out=0.577 Acc=0.95 ---230886.53 ms/batch
Step 00061900 L_out=0.325 Acc=0.89 ---254770.69 ms/batch
Step 00061950 L_out=0.242 Acc=0.95 ---278679.04 ms/batch
Step 00062000 L_out=0.306 Acc=0.95 ---302287.66 ms/batch
Round 5 | epoch=2, train costTime=317.02632570266724, | total_activation_sum=30242102
Round 5 | ****EPOCH 3****
Step 00062050 L_out=0.678 Acc=0.92 --- 9098.25 ms/batch
Step 00062100 L_out=0.432 Acc=0.89 ---32808.91 ms/batch
Step 00062150 L_out=0.384 Acc=0.92 ---56683.02 ms/batch
Step 00062200 L_out=0.875 Acc=0.93 ---80190.32 ms/batch
Step 00062250 L_out=0.553 Acc=0.82 ---103838.46 ms/batch
Step 00062300 L_out=0.543 Acc=0.91 ---127582.84 ms/batch
Step 00062350 L_out=0.382 Acc=0.90 ---151164.17 ms/batch
Step 00062400 L_out=0.424 Acc=0.91 ---174971.20 ms/batch
Step 00062450 L_out=0.672 Acc=0.84 ---198674.60 ms/batch
Step 00062500 L_out=0.221 Acc=0.97 ---222450.98 ms/batch
Step 00062550 L_out=0.557 Acc=0.91 ---246285.96 ms/batch
Step 00062600 L_out=0.069 Acc=0.95 ---270046.68 ms/batch
Step 00062650 L_out=0.250 Acc=0.93 ---293764.75 ms/batch
Round 5 | epoch=3, train costTime=316.5993103981018, | total_activation_sum=29995470
Round 5 | ****EPOCH 4****
Step 00062700 L_out=0.096 Acc=0.97 ---  950.72 ms/batch
Step 00062750 L_out=0.237 Acc=0.93 ---24559.47 ms/batch
Step 00062800 L_out=0.415 Acc=0.90 ---48485.48 ms/batch
Step 00062850 L_out=0.566 Acc=0.89 ---72259.51 ms/batch
Step 00062900 L_out=0.759 Acc=0.84 ---96119.88 ms/batch
Step 00062950 L_out=0.303 Acc=0.93 ---119688.65 ms/batch
Step 00063000 L_out=0.389 Acc=0.87 ---143503.47 ms/batch
Step 00063050 L_out=0.293 Acc=0.92 ---167582.42 ms/batch
Step 00063100 L_out=0.198 Acc=0.94 ---191662.53 ms/batch
Step 00063150 L_out=0.480 Acc=0.92 ---215477.39 ms/batch
Step 00063200 L_out=0.645 Acc=0.90 ---239132.94 ms/batch
Step 00063250 L_out=0.542 Acc=0.88 ---262893.97 ms/batch
Step 00063300 L_out=0.277 Acc=0.92 ---286539.61 ms/batch
Step 00063350 L_out=0.556 Acc=0.90 ---310114.34 ms/batch
Round 5 | epoch=4, train costTime=317.22188425064087, | total_activation_sum=30370222
Round 5 | ****EPOCH 5****
Step 00063400 L_out=1.096 Acc=0.85 ---16565.58 ms/batch
Step 00063450 L_out=0.547 Acc=0.86 ---40286.46 ms/batch
Step 00063500 L_out=0.989 Acc=0.87 ---64172.76 ms/batch
Step 00063550 L_out=0.828 Acc=0.84 ---87750.01 ms/batch
Step 00063600 L_out=0.534 Acc=0.95 ---111450.39 ms/batch
Step 00063650 L_out=0.086 Acc=0.96 ---134869.27 ms/batch
Step 00063700 L_out=0.447 Acc=0.91 ---158465.70 ms/batch
Step 00063750 L_out=0.342 Acc=0.95 ---182063.17 ms/batch
Step 00063800 L_out=0.261 Acc=0.95 ---205917.52 ms/batch
Step 00063850 L_out=0.839 Acc=0.89 ---229746.95 ms/batch
Step 00063900 L_out=0.599 Acc=0.91 ---253473.75 ms/batch
Step 00063950 L_out=0.502 Acc=0.92 ---277263.52 ms/batch
Step 00064000 L_out=0.786 Acc=0.93 ---300883.90 ms/batch
Round 5 | epoch=5, train costTime=315.87136936187744, | total_activation_sum=30143855
Round 5 | ****EPOCH 6****
Step 00064050 L_out=0.674 Acc=0.93 --- 8498.67 ms/batch
Step 00064100 L_out=0.070 Acc=0.98 ---32242.40 ms/batch
Step 00064150 L_out=0.640 Acc=0.87 ---55860.56 ms/batch
Step 00064200 L_out=0.304 Acc=0.97 ---79785.60 ms/batch
Step 00064250 L_out=0.300 Acc=0.94 ---103443.16 ms/batch
Step 00064300 L_out=0.305 Acc=0.95 ---127123.67 ms/batch
Step 00064350 L_out=0.302 Acc=0.95 ---150818.71 ms/batch
Step 00064400 L_out=0.263 Acc=0.91 ---174699.67 ms/batch
Step 00064450 L_out=0.147 Acc=0.95 ---198285.76 ms/batch
Step 00064500 L_out=0.256 Acc=0.95 ---222050.29 ms/batch
Step 00064550 L_out=0.887 Acc=0.92 ---245695.85 ms/batch
Step 00064600 L_out=0.368 Acc=0.92 ---269412.06 ms/batch
Step 00064650 L_out=0.423 Acc=0.87 ---293118.10 ms/batch
Round 5 | epoch=6, train costTime=316.4905107021332, | total_activation_sum=30349754
Round 5 | ****EPOCH 7****
Step 00064700 L_out=0.329 Acc=0.94 ---  475.03 ms/batch
Step 00064750 L_out=0.184 Acc=0.98 ---24237.17 ms/batch
Step 00064800 L_out=0.229 Acc=0.95 ---48082.21 ms/batch
Step 00064850 L_out=0.394 Acc=0.91 ---72077.32 ms/batch
Step 00064900 L_out=0.167 Acc=0.94 ---96021.25 ms/batch
Step 00064950 L_out=0.312 Acc=0.92 ---119777.98 ms/batch
Step 00065000 L_out=0.529 Acc=0.91 ---143520.14 ms/batch
Step 00065050 L_out=0.855 Acc=0.86 ---167335.49 ms/batch
Step 00065100 L_out=0.359 Acc=0.93 ---191220.91 ms/batch
Step 00065150 L_out=0.358 Acc=0.89 ---214996.10 ms/batch
Step 00065200 L_out=0.694 Acc=0.85 ---238674.18 ms/batch
Step 00065250 L_out=0.129 Acc=0.97 ---262320.55 ms/batch
Step 00065300 L_out=0.328 Acc=0.92 ---285735.16 ms/batch
Step 00065350 L_out=0.648 Acc=0.88 ---309381.74 ms/batch
Round 5 | epoch=7, train costTime=316.9189386367798, | total_activation_sum=30109175
Round 5 | ****EPOCH 8****
Step 00065400 L_out=0.171 Acc=0.96 ---16102.87 ms/batch
Step 00065450 L_out=0.373 Acc=0.92 ---39744.69 ms/batch
Step 00065500 L_out=0.532 Acc=0.91 ---63632.01 ms/batch
Step 00065550 L_out=0.315 Acc=0.96 ---87321.47 ms/batch
Step 00065600 L_out=0.277 Acc=0.95 ---111216.62 ms/batch
Step 00065650 L_out=0.356 Acc=0.91 ---134874.64 ms/batch
Step 00065700 L_out=0.216 Acc=0.94 ---158564.03 ms/batch
Step 00065750 L_out=0.445 Acc=0.95 ---182115.38 ms/batch
Step 00065800 L_out=0.316 Acc=0.89 ---205981.78 ms/batch
Step 00065850 L_out=0.198 Acc=0.95 ---229650.42 ms/batch
Step 00065900 L_out=0.611 Acc=0.87 ---253298.02 ms/batch
Step 00065950 L_out=0.278 Acc=0.96 ---276880.27 ms/batch
Step 00066000 L_out=0.259 Acc=0.92 ---300801.61 ms/batch
Round 5 | epoch=8, train costTime=316.286972284317, | total_activation_sum=30169917
Round 5 | ****EPOCH 9****
Step 00066050 L_out=0.324 Acc=0.93 --- 8022.19 ms/batch
Step 00066100 L_out=0.275 Acc=0.97 ---31630.85 ms/batch
Step 00066150 L_out=0.362 Acc=0.90 ---55626.24 ms/batch
Step 00066200 L_out=0.861 Acc=0.89 ---79202.86 ms/batch
Step 00066250 L_out=0.563 Acc=0.92 ---102667.05 ms/batch
Step 00066300 L_out=0.362 Acc=0.95 ---126389.60 ms/batch
Step 00066350 L_out=0.237 Acc=0.96 ---150208.16 ms/batch
Step 00066400 L_out=0.363 Acc=0.91 ---173904.81 ms/batch
Step 00066450 L_out=0.169 Acc=0.95 ---197369.47 ms/batch
Step 00066500 L_out=0.257 Acc=0.94 ---221143.55 ms/batch
Step 00066550 L_out=0.353 Acc=0.90 ---244926.42 ms/batch
Step 00066600 L_out=0.257 Acc=0.96 ---268538.69 ms/batch
Step 00066650 L_out=0.334 Acc=0.90 ---292278.67 ms/batch
Step 00066700 L_out=0.336 Acc=0.92 ---315853.13 ms/batch
Round 5 | epoch=9, train costTime=315.8534336090088, | total_activation_sum=30115188
Round 5 | ****EPOCH 10****
Step 00066750 L_out=0.266 Acc=0.95 ---23475.78 ms/batch
Step 00066800 L_out=0.132 Acc=0.96 ---46965.42 ms/batch
Step 00066850 L_out=0.349 Acc=0.95 ---70573.31 ms/batch
Step 00066900 L_out=0.403 Acc=0.93 ---94387.48 ms/batch
Step 00066950 L_out=0.293 Acc=0.93 ---118086.92 ms/batch
Step 00067000 L_out=0.172 Acc=0.98 ---141814.52 ms/batch
Step 00067050 L_out=0.273 Acc=0.94 ---165332.74 ms/batch
Step 00067100 L_out=0.251 Acc=0.95 ---189089.49 ms/batch
Step 00067150 L_out=0.072 Acc=0.97 ---212826.85 ms/batch
Step 00067200 L_out=0.457 Acc=0.92 ---236670.26 ms/batch
Step 00067250 L_out=0.500 Acc=0.93 ---260312.01 ms/batch
Step 00067300 L_out=0.481 Acc=0.91 ---284003.67 ms/batch
Step 00067350 L_out=0.603 Acc=0.92 ---307792.23 ms/batch
Round 5 | epoch=10, train costTime=315.78485584259033, | total_activation_sum=30273326
Round 5 | ****EPOCH 11****
Step 00067400 L_out=0.321 Acc=0.94 ---15682.77 ms/batch
Step 00067450 L_out=0.317 Acc=0.94 ---39507.07 ms/batch
Step 00067500 L_out=0.164 Acc=0.96 ---63303.73 ms/batch
Step 00067550 L_out=0.319 Acc=0.94 ---87138.50 ms/batch
Step 00067600 L_out=0.364 Acc=0.96 ---110705.89 ms/batch
Step 00067650 L_out=0.250 Acc=0.93 ---134399.75 ms/batch
Step 00067700 L_out=0.279 Acc=0.95 ---157947.65 ms/batch
Step 00067750 L_out=0.356 Acc=0.94 ---181507.15 ms/batch
Step 00067800 L_out=0.082 Acc=0.98 ---205252.65 ms/batch
Step 00067850 L_out=0.059 Acc=0.94 ---228934.01 ms/batch
Step 00067900 L_out=0.253 Acc=0.94 ---252655.87 ms/batch
Step 00067950 L_out=1.121 Acc=0.94 ---276320.88 ms/batch
Step 00068000 L_out=0.156 Acc=0.96 ---300036.24 ms/batch
Round 5 | epoch=11, train costTime=316.11711835861206, | total_activation_sum=30210059
Round 5 | ****EPOCH 12****
Step 00068050 L_out=0.253 Acc=0.94 --- 7468.79 ms/batch
Step 00068100 L_out=0.475 Acc=0.88 ---31152.16 ms/batch
Step 00068150 L_out=0.097 Acc=0.98 ---54911.40 ms/batch
Step 00068200 L_out=0.585 Acc=0.81 ---78558.07 ms/batch
Step 00068250 L_out=0.187 Acc=0.98 ---102285.87 ms/batch
Step 00068300 L_out=0.203 Acc=0.95 ---125837.91 ms/batch
Step 00068350 L_out=0.281 Acc=0.91 ---149383.64 ms/batch
Step 00068400 L_out=0.554 Acc=0.94 ---173129.91 ms/batch
Step 00068450 L_out=0.155 Acc=0.95 ---196829.80 ms/batch
Step 00068500 L_out=0.263 Acc=0.94 ---220382.17 ms/batch
Step 00068550 L_out=0.278 Acc=0.92 ---243915.67 ms/batch
Step 00068600 L_out=0.710 Acc=0.89 ---267750.38 ms/batch
Step 00068650 L_out=0.552 Acc=0.91 ---291654.08 ms/batch
Step 00068700 L_out=0.247 Acc=0.94 ---315519.07 ms/batch
Round 5 | epoch=12, train costTime=315.99760365486145, | total_activation_sum=30213734
Round 5 | ****EPOCH 13****
Step 00068750 L_out=0.320 Acc=0.93 ---23137.08 ms/batch
Step 00068800 L_out=0.227 Acc=0.95 ---46902.57 ms/batch
Step 00068850 L_out=0.610 Acc=0.91 ---70549.73 ms/batch
Step 00068900 L_out=0.331 Acc=0.92 ---94457.68 ms/batch
Step 00068950 L_out=0.190 Acc=0.94 ---118053.94 ms/batch
Step 00069000 L_out=0.162 Acc=0.94 ---141863.39 ms/batch
Step 00069050 L_out=0.142 Acc=0.97 ---165525.23 ms/batch
Step 00069100 L_out=0.359 Acc=0.93 ---189315.61 ms/batch
Step 00069150 L_out=0.170 Acc=0.94 ---212967.89 ms/batch
Step 00069200 L_out=0.115 Acc=0.97 ---236651.56 ms/batch
Step 00069250 L_out=0.070 Acc=0.97 ---260354.49 ms/batch
Step 00069300 L_out=0.168 Acc=0.95 ---283944.37 ms/batch
Step 00069350 L_out=0.167 Acc=0.95 ---307572.78 ms/batch
Round 5 | epoch=13, train costTime=316.15381503105164, | total_activation_sum=30325524
Round 5 | ****EPOCH 14****
Step 00069400 L_out=0.182 Acc=0.95 ---15037.03 ms/batch
Step 00069450 L_out=0.331 Acc=0.91 ---38598.22 ms/batch
Step 00069500 L_out=0.353 Acc=0.92 ---62407.05 ms/batch
Step 00069550 L_out=0.287 Acc=0.92 ---85940.50 ms/batch
Step 00069600 L_out=0.482 Acc=0.94 ---109448.24 ms/batch
Step 00069650 L_out=0.189 Acc=0.85 ---133097.59 ms/batch
Step 00069700 L_out=0.312 Acc=0.95 ---156797.63 ms/batch
Step 00069750 L_out=0.274 Acc=0.93 ---180541.17 ms/batch
Step 00069800 L_out=0.224 Acc=0.94 ---204118.23 ms/batch
Step 00069850 L_out=0.462 Acc=0.93 ---227886.62 ms/batch
Step 00069900 L_out=0.095 Acc=0.97 ---251695.40 ms/batch
Step 00069950 L_out=0.630 Acc=0.88 ---275562.52 ms/batch
Step 00070000 L_out=0.176 Acc=0.95 ---299312.61 ms/batch
Round 5 | epoch=14, train costTime=315.72652673721313, | total_activation_sum=30045504
Round 5 | ****EPOCH 15****
Step 00070050 L_out=0.146 Acc=0.94 --- 7007.77 ms/batch
Step 00070100 L_out=0.552 Acc=0.90 ---30722.04 ms/batch
Step 00070150 L_out=0.447 Acc=0.89 ---54556.68 ms/batch
Step 00070200 L_out=0.349 Acc=0.94 ---78192.79 ms/batch
Step 00070250 L_out=0.349 Acc=0.96 ---101748.87 ms/batch
Step 00070300 L_out=0.456 Acc=0.94 ---125202.92 ms/batch
Step 00070350 L_out=0.383 Acc=0.93 ---148917.41 ms/batch
Step 00070400 L_out=0.436 Acc=0.93 ---172774.66 ms/batch
Step 00070450 L_out=0.228 Acc=0.95 ---196628.42 ms/batch
Step 00070500 L_out=0.315 Acc=0.90 ---220396.47 ms/batch
Step 00070550 L_out=0.175 Acc=0.96 ---243823.35 ms/batch
Step 00070600 L_out=0.176 Acc=0.97 ---267576.69 ms/batch
Step 00070650 L_out=0.094 Acc=0.98 ---291240.94 ms/batch
Step 00070700 L_out=0.299 Acc=0.90 ---315005.39 ms/batch
Round 5 | epoch=15, train costTime=315.9507884979248, | total_activation_sum=30401352
Round 5 | ****EPOCH 16****
Step 00070750 L_out=0.195 Acc=0.95 ---22809.94 ms/batch
Step 00070800 L_out=0.202 Acc=0.96 ---46443.69 ms/batch
Step 00070850 L_out=0.462 Acc=0.95 ---70177.88 ms/batch
Step 00070900 L_out=0.145 Acc=0.95 ---93848.76 ms/batch
Step 00070950 L_out=0.400 Acc=0.92 ---117616.65 ms/batch
Step 00071000 L_out=0.271 Acc=0.95 ---141529.50 ms/batch
Step 00071050 L_out=0.606 Acc=0.92 ---164790.84 ms/batch
Step 00071100 L_out=0.196 Acc=0.95 ---188794.65 ms/batch
Step 00071150 L_out=0.065 Acc=0.97 ---212361.56 ms/batch
Step 00071200 L_out=0.451 Acc=0.89 ---235942.41 ms/batch
Step 00071250 L_out=0.082 Acc=0.97 ---259667.54 ms/batch
Step 00071300 L_out=0.345 Acc=0.96 ---283149.87 ms/batch
Step 00071350 L_out=0.278 Acc=0.93 ---306842.08 ms/batch
Round 5 | epoch=16, train costTime=315.70587182044983, | total_activation_sum=30179010
Round 5 | ****EPOCH 17****
Step 00071400 L_out=0.179 Acc=0.97 ---14630.38 ms/batch
Step 00071450 L_out=0.203 Acc=0.93 ---38403.54 ms/batch
Step 00071500 L_out=0.130 Acc=0.97 ---62161.43 ms/batch
Step 00071550 L_out=0.107 Acc=0.96 ---85694.74 ms/batch
Step 00071600 L_out=0.172 Acc=0.92 ---109369.52 ms/batch
Step 00071650 L_out=0.190 Acc=0.94 ---133029.70 ms/batch
Step 00071700 L_out=0.518 Acc=0.85 ---156612.99 ms/batch
Step 00071750 L_out=0.212 Acc=0.98 ---180206.20 ms/batch
Step 00071800 L_out=0.146 Acc=0.97 ---203987.53 ms/batch
Step 00071850 L_out=0.096 Acc=0.96 ---227528.48 ms/batch
Step 00071900 L_out=0.104 Acc=0.97 ---251114.91 ms/batch
Step 00071950 L_out=0.222 Acc=0.94 ---274933.93 ms/batch
Step 00072000 L_out=0.310 Acc=0.96 ---298710.39 ms/batch
Round 5 | epoch=17, train costTime=315.75960540771484, | total_activation_sum=30381001
Round 5 | epoch=18, current m_iou=0.694245458097257
Round 5 | Best m_IoU is: 0.694, OA is: 0.916 | val costTime=118.5469274520874
Round 5 | ****EPOCH 18****
Step 00072050 L_out=0.133 Acc=0.98 --- 6463.07 ms/batch
Step 00072100 L_out=0.281 Acc=0.92 ---29889.61 ms/batch
Step 00072150 L_out=0.479 Acc=0.95 ---53526.07 ms/batch
Step 00072200 L_out=0.433 Acc=0.91 ---76665.48 ms/batch
Step 00072250 L_out=0.372 Acc=0.92 ---100179.67 ms/batch
Step 00072300 L_out=0.406 Acc=0.92 ---123823.32 ms/batch
Step 00072350 L_out=0.104 Acc=0.97 ---147547.45 ms/batch
Step 00072400 L_out=0.154 Acc=0.95 ---171071.82 ms/batch
Step 00072450 L_out=0.789 Acc=0.89 ---194696.90 ms/batch
Step 00072500 L_out=0.262 Acc=0.92 ---218428.33 ms/batch
Step 00072550 L_out=0.150 Acc=0.94 ---242040.03 ms/batch
Step 00072600 L_out=0.132 Acc=0.95 ---265618.24 ms/batch
Step 00072650 L_out=0.098 Acc=0.97 ---289116.70 ms/batch
Step 00072700 L_out=0.150 Acc=0.96 ---312578.75 ms/batch
Round 5 | epoch=18, train costTime=314.00274777412415, | total_activation_sum=30144004
Round 5 | epoch=19, current m_iou=0.7035653061227378
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.88812065124512
Round 5 | ****EPOCH 19****
Step 00072750 L_out=0.121 Acc=0.96 ---22241.22 ms/batch
Step 00072800 L_out=0.161 Acc=0.94 ---46045.57 ms/batch
Step 00072850 L_out=0.071 Acc=0.96 ---69672.73 ms/batch
Step 00072900 L_out=0.102 Acc=0.97 ---93335.76 ms/batch
Step 00072950 L_out=0.131 Acc=0.98 ---117152.88 ms/batch
Step 00073000 L_out=0.101 Acc=0.97 ---140769.98 ms/batch
Step 00073050 L_out=0.122 Acc=0.95 ---164232.20 ms/batch
Step 00073100 L_out=0.176 Acc=0.95 ---187784.01 ms/batch
Step 00073150 L_out=0.533 Acc=0.90 ---211499.12 ms/batch
Step 00073200 L_out=0.294 Acc=0.96 ---235281.48 ms/batch
Step 00073250 L_out=0.236 Acc=0.97 ---258845.11 ms/batch
Step 00073300 L_out=0.390 Acc=0.92 ---282574.99 ms/batch
Step 00073350 L_out=0.520 Acc=0.89 ---306121.00 ms/batch
Round 5 | epoch=19, train costTime=315.667991399765, | total_activation_sum=30182531
Round 5 | epoch=20, current m_iou=0.7026223664252654
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.08225727081299
Round 5 | ****EPOCH 20****
Step 00073400 L_out=0.318 Acc=0.94 ---14289.31 ms/batch
Step 00073450 L_out=0.384 Acc=0.92 ---37958.13 ms/batch
Step 00073500 L_out=0.313 Acc=0.97 ---61482.25 ms/batch
Step 00073550 L_out=0.174 Acc=0.97 ---85232.09 ms/batch
Step 00073600 L_out=0.194 Acc=0.93 ---108847.15 ms/batch
Step 00073650 L_out=0.158 Acc=0.91 ---132521.13 ms/batch
Step 00073700 L_out=0.223 Acc=0.96 ---156320.04 ms/batch
Step 00073750 L_out=0.249 Acc=0.95 ---179800.99 ms/batch
Step 00073800 L_out=0.225 Acc=0.95 ---203501.57 ms/batch
Step 00073850 L_out=0.272 Acc=0.94 ---227048.13 ms/batch
Step 00073900 L_out=0.292 Acc=0.93 ---250703.80 ms/batch
Step 00073950 L_out=0.437 Acc=0.91 ---274251.85 ms/batch
Step 00074000 L_out=0.118 Acc=0.96 ---298073.76 ms/batch
Round 5 | epoch=20, train costTime=315.44137930870056, | total_activation_sum=30170761
Round 5 | epoch=21, current m_iou=0.6916762351145902
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.85209703445435
Round 5 | ****EPOCH 21****
Step 00074050 L_out=0.120 Acc=0.97 --- 6293.07 ms/batch
Step 00074100 L_out=0.093 Acc=0.97 ---30058.40 ms/batch
Step 00074150 L_out=0.564 Acc=0.92 ---53642.98 ms/batch
Step 00074200 L_out=0.253 Acc=0.93 ---77514.21 ms/batch
Step 00074250 L_out=0.401 Acc=0.93 ---101363.82 ms/batch
Step 00074300 L_out=0.647 Acc=0.88 ---125248.52 ms/batch
Step 00074350 L_out=0.368 Acc=0.91 ---148845.60 ms/batch
Step 00074400 L_out=0.565 Acc=0.93 ---172611.20 ms/batch
Step 00074450 L_out=0.197 Acc=0.94 ---196150.74 ms/batch
Step 00074500 L_out=0.629 Acc=0.87 ---219953.86 ms/batch
Step 00074550 L_out=0.257 Acc=0.92 ---243626.34 ms/batch
Step 00074600 L_out=0.238 Acc=0.94 ---267474.44 ms/batch
Step 00074650 L_out=0.207 Acc=0.95 ---290978.59 ms/batch
Step 00074700 L_out=0.187 Acc=0.97 ---314726.32 ms/batch
Round 5 | epoch=21, train costTime=316.5861804485321, | total_activation_sum=30325148
Round 5 | epoch=22, current m_iou=0.6954212576760408
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.52445244789124
Round 5 | ****EPOCH 22****
Step 00074750 L_out=0.195 Acc=0.95 ---21624.96 ms/batch
Step 00074800 L_out=0.243 Acc=0.92 ---45257.66 ms/batch
Step 00074850 L_out=0.177 Acc=0.97 ---68988.50 ms/batch
Step 00074900 L_out=0.224 Acc=0.95 ---92776.91 ms/batch
Step 00074950 L_out=0.323 Acc=0.95 ---116495.93 ms/batch
Step 00075000 L_out=0.214 Acc=0.96 ---140007.27 ms/batch
Step 00075050 L_out=0.359 Acc=0.95 ---163841.21 ms/batch
Step 00075100 L_out=0.180 Acc=0.96 ---187425.32 ms/batch
Step 00075150 L_out=0.175 Acc=0.95 ---211230.08 ms/batch
Step 00075200 L_out=0.096 Acc=0.96 ---235033.51 ms/batch
Step 00075250 L_out=0.301 Acc=0.92 ---258524.71 ms/batch
Step 00075300 L_out=0.226 Acc=0.97 ---282322.11 ms/batch
Step 00075350 L_out=0.409 Acc=0.96 ---306165.14 ms/batch
Round 5 | epoch=22, train costTime=316.2479145526886, | total_activation_sum=30251274
Round 5 | epoch=23, current m_iou=0.6938163275101964
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.09045267105103
Round 5 | ****EPOCH 23****
Step 00075400 L_out=0.156 Acc=0.95 ---13655.76 ms/batch
Step 00075450 L_out=0.259 Acc=0.93 ---37383.82 ms/batch
Step 00075500 L_out=0.153 Acc=0.96 ---61142.52 ms/batch
Step 00075550 L_out=0.165 Acc=0.94 ---84726.45 ms/batch
Step 00075600 L_out=0.166 Acc=0.94 ---108421.60 ms/batch
Step 00075650 L_out=0.278 Acc=0.95 ---132292.31 ms/batch
Step 00075700 L_out=0.209 Acc=0.96 ---155833.11 ms/batch
Step 00075750 L_out=0.133 Acc=0.93 ---179792.56 ms/batch
Step 00075800 L_out=0.293 Acc=0.91 ---203364.21 ms/batch
Step 00075850 L_out=0.132 Acc=0.96 ---227040.94 ms/batch
Step 00075900 L_out=0.330 Acc=0.94 ---250691.37 ms/batch
Step 00075950 L_out=0.157 Acc=0.96 ---274288.96 ms/batch
Step 00076000 L_out=0.209 Acc=0.96 ---297942.32 ms/batch
Round 5 | epoch=23, train costTime=315.8684983253479, | total_activation_sum=30128905
Round 5 | epoch=24, current m_iou=0.6964806275805672
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.87111759185791
Round 5 | ****EPOCH 24****
Step 00076050 L_out=0.330 Acc=0.95 --- 5642.97 ms/batch
Step 00076100 L_out=0.156 Acc=0.97 ---29372.32 ms/batch
Step 00076150 L_out=0.166 Acc=0.97 ---53136.71 ms/batch
Step 00076200 L_out=0.104 Acc=0.97 ---76710.91 ms/batch
Step 00076250 L_out=0.279 Acc=0.90 ---100240.67 ms/batch
Step 00076300 L_out=0.329 Acc=0.94 ---123978.59 ms/batch
Step 00076350 L_out=0.148 Acc=0.96 ---147695.78 ms/batch
Step 00076400 L_out=0.102 Acc=0.98 ---171585.90 ms/batch
Step 00076450 L_out=0.363 Acc=0.91 ---195104.84 ms/batch
Step 00076500 L_out=0.136 Acc=0.96 ---218678.51 ms/batch
Step 00076550 L_out=0.264 Acc=0.91 ---242309.64 ms/batch
Step 00076600 L_out=0.127 Acc=0.96 ---265723.17 ms/batch
Step 00076650 L_out=0.131 Acc=0.97 ---289365.96 ms/batch
Step 00076700 L_out=0.152 Acc=0.93 ---313031.48 ms/batch
Round 5 | epoch=24, train costTime=315.347852230072, | total_activation_sum=30127847
Round 5 | epoch=25, current m_iou=0.695686880418574
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.57841563224792
Round 5 | ****EPOCH 25****
Step 00076750 L_out=0.285 Acc=0.96 ---21277.27 ms/batch
Step 00076800 L_out=0.275 Acc=0.94 ---45032.26 ms/batch
Step 00076850 L_out=0.172 Acc=0.92 ---68541.67 ms/batch
Step 00076900 L_out=0.134 Acc=0.92 ---92268.37 ms/batch
Step 00076950 L_out=0.068 Acc=0.96 ---115868.72 ms/batch
Step 00077000 L_out=0.095 Acc=0.98 ---139576.80 ms/batch
Step 00077050 L_out=0.127 Acc=0.96 ---163118.89 ms/batch
Step 00077100 L_out=0.095 Acc=0.97 ---186804.73 ms/batch
Step 00077150 L_out=0.099 Acc=0.96 ---210508.81 ms/batch
Step 00077200 L_out=0.093 Acc=0.97 ---234192.88 ms/batch
Step 00077250 L_out=0.388 Acc=0.93 ---258113.58 ms/batch
Step 00077300 L_out=0.202 Acc=0.95 ---281821.61 ms/batch
Step 00077350 L_out=0.067 Acc=0.97 ---305225.01 ms/batch
Round 5 | epoch=25, train costTime=315.65196561813354, | total_activation_sum=30460746
Round 5 | epoch=26, current m_iou=0.698962548610285
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.74489164352417
Round 5 | ****EPOCH 26****
Step 00077400 L_out=0.121 Acc=0.96 ---13162.82 ms/batch
Step 00077450 L_out=0.077 Acc=0.96 ---36648.86 ms/batch
Step 00077500 L_out=0.186 Acc=0.96 ---60048.69 ms/batch
Step 00077550 L_out=0.466 Acc=0.87 ---83640.06 ms/batch
Step 00077600 L_out=0.162 Acc=0.94 ---107424.38 ms/batch
Step 00077650 L_out=0.272 Acc=0.94 ---131085.39 ms/batch
Step 00077700 L_out=0.067 Acc=0.97 ---154811.90 ms/batch
Step 00077750 L_out=0.163 Acc=0.97 ---178471.27 ms/batch
Step 00077800 L_out=0.138 Acc=0.97 ---202041.55 ms/batch
Step 00077850 L_out=0.449 Acc=0.94 ---225666.02 ms/batch
Step 00077900 L_out=0.076 Acc=0.99 ---248752.50 ms/batch
Step 00077950 L_out=0.092 Acc=0.97 ---272418.58 ms/batch
Step 00078000 L_out=0.327 Acc=0.93 ---296150.44 ms/batch
Round 5 | epoch=26, train costTime=314.74005460739136, | total_activation_sum=30402543
Round 5 | epoch=27, current m_iou=0.6977053251610237
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.36773133277893
Round 5 | ****EPOCH 27****
Step 00078050 L_out=0.283 Acc=0.91 --- 5097.47 ms/batch
Step 00078100 L_out=0.161 Acc=0.95 ---28742.86 ms/batch
Step 00078150 L_out=0.117 Acc=0.93 ---52605.67 ms/batch
Step 00078200 L_out=0.448 Acc=0.90 ---76327.37 ms/batch
Step 00078250 L_out=0.151 Acc=0.97 ---100035.45 ms/batch
Step 00078300 L_out=0.250 Acc=0.96 ---123930.91 ms/batch
Step 00078350 L_out=0.146 Acc=0.97 ---147643.78 ms/batch
Step 00078400 L_out=0.199 Acc=0.95 ---171459.36 ms/batch
Step 00078450 L_out=0.331 Acc=0.95 ---195009.06 ms/batch
Step 00078500 L_out=0.188 Acc=0.94 ---218609.42 ms/batch
Step 00078550 L_out=0.167 Acc=0.96 ---242196.48 ms/batch
Step 00078600 L_out=0.146 Acc=0.96 ---265908.33 ms/batch
Step 00078650 L_out=0.191 Acc=0.96 ---289667.43 ms/batch
Step 00078700 L_out=0.255 Acc=0.95 ---313396.45 ms/batch
Round 5 | epoch=27, train costTime=316.22504687309265, | total_activation_sum=29909113
Round 5 | epoch=28, current m_iou=0.6951208628978236
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.24599552154541
Round 5 | ****EPOCH 28****
Step 00078750 L_out=0.134 Acc=0.96 ---20642.35 ms/batch
Step 00078800 L_out=0.153 Acc=0.96 ---44510.78 ms/batch
Step 00078850 L_out=0.315 Acc=0.94 ---68123.78 ms/batch
Step 00078900 L_out=0.167 Acc=0.96 ---91648.51 ms/batch
Step 00078950 L_out=0.317 Acc=0.89 ---115313.12 ms/batch
Step 00079000 L_out=0.230 Acc=0.93 ---138649.76 ms/batch
Step 00079050 L_out=1.245 Acc=0.90 ---162284.23 ms/batch
Step 00079100 L_out=0.139 Acc=0.97 ---185934.62 ms/batch
Step 00079150 L_out=0.175 Acc=0.90 ---209646.05 ms/batch
Step 00079200 L_out=0.240 Acc=0.95 ---233479.30 ms/batch
Step 00079250 L_out=0.134 Acc=0.98 ---256988.20 ms/batch
Step 00079300 L_out=0.183 Acc=0.96 ---280581.03 ms/batch
Step 00079350 L_out=0.183 Acc=0.94 ---304137.69 ms/batch
Round 5 | epoch=28, train costTime=315.02067732810974, | total_activation_sum=30198453
Round 5 | epoch=29, current m_iou=0.6980466373693177
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=118.3519024848938
Round 5 | ****EPOCH 29****
Step 00079400 L_out=0.161 Acc=0.97 ---12777.74 ms/batch
Step 00079450 L_out=0.184 Acc=0.95 ---36482.46 ms/batch
Step 00079500 L_out=0.179 Acc=0.95 ---59939.39 ms/batch
Step 00079550 L_out=0.091 Acc=0.97 ---83929.26 ms/batch
Step 00079600 L_out=0.328 Acc=0.94 ---107577.71 ms/batch
Step 00079650 L_out=0.419 Acc=0.94 ---131227.75 ms/batch
Step 00079700 L_out=0.320 Acc=0.95 ---154916.09 ms/batch
Step 00079750 L_out=0.198 Acc=0.96 ---178626.53 ms/batch
Step 00079800 L_out=0.253 Acc=0.96 ---202300.03 ms/batch
Step 00079850 L_out=0.203 Acc=0.94 ---225958.89 ms/batch
Step 00079900 L_out=0.061 Acc=0.97 ---249640.08 ms/batch
Step 00079950 L_out=0.221 Acc=0.95 ---272722.88 ms/batch
Step 00080000 L_out=0.153 Acc=0.96 ---296426.65 ms/batch
Round 5 | epoch=29, train costTime=315.1035957336426, | total_activation_sum=30107124
Round 5 | epoch=30, current m_iou=0.6940997174658068
Round 5 | Best m_IoU is: 0.704, OA is: 0.922 | val costTime=117.96047329902649
Round 5 | ****EPOCH 30****
Round 6 | ****EPOCH 0****
Step 00080050 L_out=0.170 Acc=0.98 --- 4447.18 ms/batch
Step 00080100 L_out=1.223 Acc=0.66 ---26811.87 ms/batch
Step 00080150 L_out=1.013 Acc=0.57 ---49267.51 ms/batch
Step 00080200 L_out=1.948 Acc=0.31 ---71454.78 ms/batch
Step 00080250 L_out=0.593 Acc=0.72 ---93849.03 ms/batch
Step 00080300 L_out=0.457 Acc=0.88 ---116016.96 ms/batch
Step 00080350 L_out=0.775 Acc=0.81 ---138530.74 ms/batch
Step 00080400 L_out=0.494 Acc=0.89 ---160870.08 ms/batch
Step 00080450 L_out=0.540 Acc=0.80 ---183188.02 ms/batch
Step 00080500 L_out=1.146 Acc=0.89 ---205564.30 ms/batch
Step 00080550 L_out=0.545 Acc=0.85 ---228084.67 ms/batch
Step 00080600 L_out=1.003 Acc=0.81 ---250327.01 ms/batch
Step 00080650 L_out=0.662 Acc=0.90 ---272577.11 ms/batch
Step 00080700 L_out=0.359 Acc=0.96 ---294938.79 ms/batch
Round 6 | epoch=0, train costTime=298.0261900424957, | total_activation_sum=35414625
Round 6 | ****EPOCH 1****
Step 00080750 L_out=1.428 Acc=0.88 ---19133.07 ms/batch
Step 00080800 L_out=0.569 Acc=0.94 ---41427.45 ms/batch
Step 00080850 L_out=0.506 Acc=0.88 ---63920.42 ms/batch
Step 00080900 L_out=0.743 Acc=0.91 ---86250.27 ms/batch
Step 00080950 L_out=0.751 Acc=0.86 ---108547.91 ms/batch
Step 00081000 L_out=0.948 Acc=0.85 ---131866.88 ms/batch
Step 00081050 L_out=0.351 Acc=0.90 ---155590.14 ms/batch
Step 00081100 L_out=0.367 Acc=0.95 ---178541.93 ms/batch
Step 00081150 L_out=0.680 Acc=0.85 ---202251.70 ms/batch
Step 00081200 L_out=0.389 Acc=0.95 ---225878.76 ms/batch
Step 00081250 L_out=0.318 Acc=0.94 ---249435.19 ms/batch
Step 00081300 L_out=0.365 Acc=0.92 ---273202.11 ms/batch
Step 00081350 L_out=0.508 Acc=0.91 ---296896.74 ms/batch
Round 6 | epoch=1, train costTime=307.7202994823456, | total_activation_sum=33816067
Round 6 | ****EPOCH 2****
Step 00081400 L_out=0.287 Acc=0.94 ---11516.35 ms/batch
Step 00081450 L_out=0.437 Acc=0.92 ---41702.22 ms/batch
Step 00081500 L_out=2.001 Acc=0.93 ---65621.02 ms/batch
Step 00081550 L_out=0.465 Acc=0.93 ---88960.04 ms/batch
Step 00081600 L_out=0.115 Acc=0.97 ---112750.22 ms/batch
Step 00081650 L_out=0.259 Acc=0.94 ---136248.49 ms/batch
Step 00081700 L_out=0.257 Acc=0.93 ---159757.38 ms/batch
Step 00081750 L_out=0.416 Acc=0.91 ---183462.90 ms/batch
Step 00081800 L_out=0.849 Acc=0.89 ---206986.19 ms/batch
Step 00081850 L_out=0.727 Acc=0.93 ---230753.60 ms/batch
Step 00081900 L_out=0.328 Acc=0.92 ---254483.62 ms/batch
Step 00081950 L_out=0.223 Acc=0.97 ---278288.59 ms/batch
Step 00082000 L_out=0.436 Acc=0.93 ---301779.78 ms/batch
Round 6 | epoch=2, train costTime=321.2969083786011, | total_activation_sum=33790476
Round 6 | ****EPOCH 3****
Step 00082050 L_out=0.872 Acc=0.84 --- 4313.67 ms/batch
Step 00082100 L_out=0.443 Acc=0.95 ---27874.20 ms/batch
Step 00082150 L_out=0.625 Acc=0.87 ---51494.15 ms/batch
Step 00082200 L_out=0.362 Acc=0.93 ---73570.81 ms/batch
Step 00082250 L_out=0.576 Acc=0.93 ---95899.75 ms/batch
Step 00082300 L_out=0.504 Acc=0.88 ---118020.47 ms/batch
Step 00082350 L_out=0.530 Acc=0.91 ---140569.53 ms/batch
Step 00082400 L_out=0.470 Acc=0.93 ---162604.88 ms/batch
Step 00082450 L_out=0.823 Acc=0.88 ---184811.47 ms/batch
Step 00082500 L_out=0.693 Acc=0.85 ---207038.55 ms/batch
Step 00082550 L_out=1.018 Acc=0.86 ---229201.44 ms/batch
Step 00082600 L_out=0.185 Acc=0.97 ---251345.37 ms/batch
Step 00082650 L_out=0.453 Acc=0.94 ---273584.14 ms/batch
Step 00082700 L_out=0.553 Acc=0.92 ---295246.49 ms/batch
Round 6 | epoch=3, train costTime=298.78417587280273, | total_activation_sum=33630021
Round 6 | ****EPOCH 4****
Step 00082750 L_out=0.517 Acc=0.88 ---18380.73 ms/batch
Step 00082800 L_out=0.146 Acc=0.96 ---42130.55 ms/batch
Step 00082850 L_out=0.845 Acc=0.91 ---65966.58 ms/batch
Step 00082900 L_out=0.599 Acc=0.92 ---89335.56 ms/batch
Step 00082950 L_out=0.219 Acc=0.96 ---111333.60 ms/batch
Step 00083000 L_out=0.414 Acc=0.96 ---133204.32 ms/batch
Step 00083050 L_out=0.136 Acc=0.94 ---155007.22 ms/batch
Step 00083100 L_out=0.386 Acc=0.94 ---176842.99 ms/batch
Step 00083150 L_out=0.123 Acc=0.97 ---198540.65 ms/batch
Step 00083200 L_out=0.394 Acc=0.95 ---220454.39 ms/batch
Step 00083250 L_out=0.488 Acc=0.87 ---242248.34 ms/batch
Step 00083300 L_out=0.137 Acc=0.97 ---264238.14 ms/batch
Step 00083350 L_out=0.684 Acc=0.90 ---285998.51 ms/batch
Round 6 | epoch=4, train costTime=296.9782831668854, | total_activation_sum=33682333
Round 6 | ****EPOCH 5****
Step 00083400 L_out=0.526 Acc=0.87 ---10907.68 ms/batch
Step 00083450 L_out=0.298 Acc=0.95 ---32871.78 ms/batch
Step 00083500 L_out=0.344 Acc=0.92 ---54722.70 ms/batch
Step 00083550 L_out=0.662 Acc=0.92 ---76278.64 ms/batch
Step 00083600 L_out=0.457 Acc=0.95 ---98155.65 ms/batch
Step 00083650 L_out=0.855 Acc=0.88 ---119963.83 ms/batch
Step 00083700 L_out=0.182 Acc=0.97 ---141717.84 ms/batch
Step 00083750 L_out=0.183 Acc=0.96 ---163778.37 ms/batch
Step 00083800 L_out=0.243 Acc=0.95 ---187232.00 ms/batch
Step 00083850 L_out=0.329 Acc=0.96 ---211164.68 ms/batch
Step 00083900 L_out=0.714 Acc=0.85 ---234862.70 ms/batch
Step 00083950 L_out=0.264 Acc=0.92 ---259065.24 ms/batch
Step 00084000 L_out=0.384 Acc=0.94 ---283038.89 ms/batch
Round 6 | epoch=5, train costTime=303.3555574417114, | total_activation_sum=33857186
Round 6 | ****EPOCH 6****
Step 00084050 L_out=0.254 Acc=0.97 --- 3742.37 ms/batch
Step 00084100 L_out=0.558 Acc=0.87 ---27932.77 ms/batch
Step 00084150 L_out=0.334 Acc=0.92 ---51935.55 ms/batch
Step 00084200 L_out=0.072 Acc=0.98 ---75966.62 ms/batch
Step 00084250 L_out=0.333 Acc=0.94 ---99763.07 ms/batch
Step 00084300 L_out=0.313 Acc=0.94 ---123699.34 ms/batch
Step 00084350 L_out=0.410 Acc=0.95 ---147925.34 ms/batch
Step 00084400 L_out=0.234 Acc=0.96 ---171783.81 ms/batch
Step 00084450 L_out=0.755 Acc=0.94 ---195512.32 ms/batch
Step 00084500 L_out=0.246 Acc=0.93 ---219522.23 ms/batch
Step 00084550 L_out=1.447 Acc=0.82 ---243166.15 ms/batch
Step 00084600 L_out=0.626 Acc=0.93 ---267198.58 ms/batch
Step 00084650 L_out=0.634 Acc=0.92 ---290890.73 ms/batch
Step 00084700 L_out=0.699 Acc=0.88 ---314863.85 ms/batch
Round 6 | epoch=6, train costTime=319.35084295272827, | total_activation_sum=33516735
Round 6 | ****EPOCH 7****
Step 00084750 L_out=0.128 Acc=0.98 ---19558.77 ms/batch
Step 00084800 L_out=0.197 Acc=0.96 ---43143.67 ms/batch
Step 00084850 L_out=0.445 Acc=0.95 ---67052.18 ms/batch
Step 00084900 L_out=0.156 Acc=0.96 ---91183.37 ms/batch
Step 00084950 L_out=0.301 Acc=0.96 ---115153.37 ms/batch
Step 00085000 L_out=0.185 Acc=0.96 ---139212.56 ms/batch
Step 00085050 L_out=0.516 Acc=0.94 ---163160.73 ms/batch
Step 00085100 L_out=0.546 Acc=0.94 ---187006.54 ms/batch
Step 00085150 L_out=0.207 Acc=0.95 ---211158.15 ms/batch
Step 00085200 L_out=0.434 Acc=0.82 ---234933.28 ms/batch
Step 00085250 L_out=0.341 Acc=0.94 ---258902.17 ms/batch
Step 00085300 L_out=0.209 Acc=0.96 ---282833.66 ms/batch
Step 00085350 L_out=0.352 Acc=0.96 ---306713.67 ms/batch
Round 6 | epoch=7, train costTime=318.9925694465637, | total_activation_sum=33745144
Round 6 | ****EPOCH 8****
Step 00085400 L_out=0.255 Acc=0.97 ---11294.55 ms/batch
Step 00085450 L_out=0.544 Acc=0.87 ---35214.87 ms/batch
Step 00085500 L_out=0.154 Acc=0.96 ---58813.59 ms/batch
Step 00085550 L_out=0.281 Acc=0.94 ---82527.54 ms/batch
Step 00085600 L_out=0.499 Acc=0.90 ---106027.58 ms/batch
Step 00085650 L_out=0.200 Acc=0.96 ---129610.48 ms/batch
Step 00085700 L_out=0.356 Acc=0.91 ---153058.12 ms/batch
Step 00085750 L_out=0.143 Acc=0.97 ---177019.03 ms/batch
Step 00085800 L_out=0.467 Acc=0.94 ---201064.31 ms/batch
Step 00085850 L_out=0.372 Acc=0.93 ---224785.92 ms/batch
Step 00085900 L_out=0.494 Acc=0.91 ---248472.84 ms/batch
Step 00085950 L_out=0.392 Acc=0.95 ---272058.25 ms/batch
Step 00086000 L_out=0.337 Acc=0.93 ---295661.82 ms/batch
Round 6 | epoch=8, train costTime=316.0714502334595, | total_activation_sum=33669614
Round 6 | ****EPOCH 9****
Step 00086050 L_out=0.106 Acc=0.97 --- 3284.58 ms/batch
Step 00086100 L_out=0.186 Acc=0.97 ---27066.50 ms/batch
Step 00086150 L_out=0.371 Acc=0.94 ---50950.15 ms/batch
Step 00086200 L_out=0.322 Acc=0.97 ---74660.66 ms/batch
Step 00086250 L_out=0.465 Acc=0.95 ---98290.20 ms/batch
Step 00086300 L_out=0.555 Acc=0.97 ---122093.73 ms/batch
Step 00086350 L_out=0.193 Acc=0.95 ---145822.54 ms/batch
Step 00086400 L_out=0.544 Acc=0.94 ---169857.24 ms/batch
Step 00086450 L_out=0.125 Acc=0.98 ---193646.74 ms/batch
Step 00086500 L_out=0.142 Acc=0.98 ---217381.39 ms/batch
Step 00086550 L_out=0.297 Acc=0.92 ---241160.30 ms/batch
Step 00086600 L_out=0.560 Acc=0.90 ---264817.26 ms/batch
Step 00086650 L_out=0.347 Acc=0.87 ---288319.05 ms/batch
Step 00086700 L_out=0.232 Acc=0.95 ---311777.73 ms/batch
Round 6 | epoch=9, train costTime=316.49157786369324, | total_activation_sum=33767597
Round 6 | ****EPOCH 10****
Step 00086750 L_out=0.406 Acc=0.93 ---18848.21 ms/batch
Step 00086800 L_out=0.157 Acc=0.96 ---42749.99 ms/batch
Step 00086850 L_out=0.134 Acc=0.97 ---66120.69 ms/batch
Step 00086900 L_out=0.365 Acc=0.95 ---90239.16 ms/batch
Step 00086950 L_out=0.435 Acc=0.96 ---114333.48 ms/batch
Step 00087000 L_out=0.140 Acc=0.96 ---138050.18 ms/batch
Step 00087050 L_out=0.095 Acc=0.97 ---162130.47 ms/batch
Step 00087100 L_out=0.609 Acc=0.92 ---185778.50 ms/batch
Step 00087150 L_out=0.161 Acc=0.95 ---209703.34 ms/batch
Step 00087200 L_out=0.308 Acc=0.96 ---233590.61 ms/batch
Step 00087250 L_out=0.284 Acc=0.95 ---257596.38 ms/batch
Step 00087300 L_out=0.201 Acc=0.95 ---281230.83 ms/batch
Step 00087350 L_out=0.270 Acc=0.95 ---305039.53 ms/batch
Round 6 | epoch=10, train costTime=317.9415752887726, | total_activation_sum=33799074
Round 6 | ****EPOCH 11****
Step 00087400 L_out=0.324 Acc=0.95 ---10952.40 ms/batch
Step 00087450 L_out=0.401 Acc=0.94 ---34802.75 ms/batch
Step 00087500 L_out=0.130 Acc=0.98 ---58638.27 ms/batch
Step 00087550 L_out=0.166 Acc=0.94 ---82245.38 ms/batch
Step 00087600 L_out=0.551 Acc=0.91 ---106232.67 ms/batch
Step 00087650 L_out=0.178 Acc=0.97 ---130113.62 ms/batch
Step 00087700 L_out=0.370 Acc=0.94 ---153851.63 ms/batch
Step 00087750 L_out=0.270 Acc=0.97 ---177935.44 ms/batch
Step 00087800 L_out=0.220 Acc=0.97 ---201572.69 ms/batch
Step 00087850 L_out=0.242 Acc=0.97 ---225490.39 ms/batch
Step 00087900 L_out=0.213 Acc=0.97 ---249378.76 ms/batch
Step 00087950 L_out=0.187 Acc=0.97 ---272939.19 ms/batch
Step 00088000 L_out=0.158 Acc=0.96 ---296904.00 ms/batch
Round 6 | epoch=11, train costTime=317.65294551849365, | total_activation_sum=33623754
Round 6 | ****EPOCH 12****
Step 00088050 L_out=0.194 Acc=0.96 --- 2917.10 ms/batch
Step 00088100 L_out=0.223 Acc=0.97 ---26811.94 ms/batch
Step 00088150 L_out=0.652 Acc=0.92 ---50811.04 ms/batch
Step 00088200 L_out=0.358 Acc=0.95 ---74179.77 ms/batch
Step 00088250 L_out=0.268 Acc=0.95 ---97781.32 ms/batch
Step 00088300 L_out=1.082 Acc=0.84 ---121465.76 ms/batch
Step 00088350 L_out=0.413 Acc=0.91 ---145122.63 ms/batch
Step 00088400 L_out=0.402 Acc=0.89 ---168989.49 ms/batch
Step 00088450 L_out=0.054 Acc=0.99 ---192785.07 ms/batch
Step 00088500 L_out=0.168 Acc=0.92 ---216555.56 ms/batch
Step 00088550 L_out=0.220 Acc=0.96 ---240106.70 ms/batch
Step 00088600 L_out=0.271 Acc=0.96 ---263618.00 ms/batch
Step 00088650 L_out=0.250 Acc=0.94 ---287433.88 ms/batch
Step 00088700 L_out=0.361 Acc=0.95 ---311147.03 ms/batch
Round 6 | epoch=12, train costTime=316.33530950546265, | total_activation_sum=33750232
Round 6 | ****EPOCH 13****
Step 00088750 L_out=0.457 Acc=0.96 ---18349.93 ms/batch
Step 00088800 L_out=0.787 Acc=0.84 ---41854.23 ms/batch
Step 00088850 L_out=0.213 Acc=0.97 ---65499.35 ms/batch
Step 00088900 L_out=0.361 Acc=0.94 ---89135.77 ms/batch
Step 00088950 L_out=0.178 Acc=0.98 ---112794.36 ms/batch
Step 00089000 L_out=0.223 Acc=0.97 ---136409.77 ms/batch
Step 00089050 L_out=0.137 Acc=0.97 ---160085.65 ms/batch
Step 00089100 L_out=0.986 Acc=0.72 ---183677.21 ms/batch
Step 00089150 L_out=0.286 Acc=0.95 ---207263.69 ms/batch
Step 00089200 L_out=0.258 Acc=0.92 ---231048.08 ms/batch
Step 00089250 L_out=0.218 Acc=0.98 ---254863.95 ms/batch
Step 00089300 L_out=0.145 Acc=0.96 ---278668.86 ms/batch
Step 00089350 L_out=1.399 Acc=0.96 ---302193.21 ms/batch
Round 6 | epoch=13, train costTime=315.45572447776794, | total_activation_sum=33798228
Round 6 | ****EPOCH 14****
Step 00089400 L_out=0.092 Acc=0.97 ---10458.90 ms/batch
Step 00089450 L_out=0.063 Acc=0.99 ---34169.42 ms/batch
Step 00089500 L_out=0.203 Acc=0.95 ---57941.67 ms/batch
Step 00089550 L_out=0.187 Acc=0.94 ---81766.59 ms/batch
Step 00089600 L_out=0.325 Acc=0.89 ---105642.69 ms/batch
Step 00089650 L_out=0.159 Acc=0.98 ---129462.64 ms/batch
Step 00089700 L_out=0.162 Acc=0.96 ---153211.94 ms/batch
Step 00089750 L_out=0.227 Acc=0.96 ---176957.19 ms/batch
Step 00089800 L_out=0.614 Acc=0.91 ---200641.57 ms/batch
Step 00089850 L_out=0.268 Acc=0.93 ---224428.96 ms/batch
Step 00089900 L_out=0.148 Acc=0.97 ---248203.58 ms/batch
Step 00089950 L_out=0.295 Acc=0.94 ---271950.54 ms/batch
Step 00090000 L_out=0.180 Acc=0.96 ---295567.78 ms/batch
Round 6 | epoch=14, train costTime=316.72878766059875, | total_activation_sum=33844067
Round 6 | ****EPOCH 15****
Step 00090050 L_out=0.153 Acc=0.97 --- 2393.43 ms/batch
Step 00090100 L_out=0.187 Acc=0.96 ---26124.03 ms/batch
Step 00090150 L_out=0.418 Acc=0.93 ---49829.16 ms/batch
Step 00090200 L_out=0.139 Acc=0.99 ---73331.82 ms/batch
Step 00090250 L_out=0.133 Acc=0.97 ---97294.69 ms/batch
Step 00090300 L_out=0.725 Acc=0.91 ---121117.15 ms/batch
Step 00090350 L_out=0.178 Acc=0.95 ---144917.13 ms/batch
Step 00090400 L_out=0.138 Acc=0.95 ---168623.92 ms/batch
Step 00090450 L_out=0.283 Acc=0.96 ---192275.35 ms/batch
Step 00090500 L_out=0.203 Acc=0.94 ---216033.51 ms/batch
Step 00090550 L_out=0.275 Acc=0.97 ---239787.46 ms/batch
Step 00090600 L_out=0.415 Acc=0.91 ---263553.24 ms/batch
Step 00090650 L_out=0.438 Acc=0.94 ---287176.50 ms/batch
Step 00090700 L_out=0.204 Acc=0.95 ---310975.89 ms/batch
Round 6 | epoch=15, train costTime=316.66017174720764, | total_activation_sum=33727910
Round 6 | ****EPOCH 16****
Step 00090750 L_out=0.115 Acc=0.97 ---18212.11 ms/batch
Step 00090800 L_out=0.121 Acc=0.98 ---42083.40 ms/batch
Step 00090850 L_out=0.232 Acc=0.97 ---65634.03 ms/batch
Step 00090900 L_out=0.300 Acc=0.94 ---89610.44 ms/batch
Step 00090950 L_out=0.171 Acc=0.97 ---113416.72 ms/batch
Step 00091000 L_out=0.319 Acc=0.95 ---137297.89 ms/batch
Step 00091050 L_out=0.255 Acc=0.93 ---161171.88 ms/batch
Step 00091100 L_out=0.327 Acc=0.95 ---184906.04 ms/batch
Step 00091150 L_out=0.307 Acc=0.92 ---208641.38 ms/batch
Step 00091200 L_out=0.178 Acc=0.97 ---232336.42 ms/batch
Step 00091250 L_out=0.249 Acc=0.95 ---255976.79 ms/batch
Step 00091300 L_out=0.271 Acc=0.96 ---279989.77 ms/batch
Step 00091350 L_out=0.484 Acc=0.92 ---303567.83 ms/batch
Round 6 | epoch=16, train costTime=317.5133385658264, | total_activation_sum=33595382
Round 6 | ****EPOCH 17****
Step 00091400 L_out=0.229 Acc=0.96 ---10014.18 ms/batch
Step 00091450 L_out=0.262 Acc=0.97 ---33653.17 ms/batch
Step 00091500 L_out=0.062 Acc=0.99 ---57277.21 ms/batch
Step 00091550 L_out=0.147 Acc=0.97 ---81076.23 ms/batch
Step 00091600 L_out=0.115 Acc=0.97 ---104938.38 ms/batch
Step 00091650 L_out=0.147 Acc=0.97 ---128414.56 ms/batch
Step 00091700 L_out=0.376 Acc=0.95 ---152112.43 ms/batch
Step 00091750 L_out=0.294 Acc=0.94 ---175982.80 ms/batch
Step 00091800 L_out=0.186 Acc=0.96 ---199788.82 ms/batch
Step 00091850 L_out=0.143 Acc=0.96 ---223369.82 ms/batch
Step 00091900 L_out=0.065 Acc=0.99 ---247041.10 ms/batch
Step 00091950 L_out=0.722 Acc=0.92 ---270792.10 ms/batch
Step 00092000 L_out=0.130 Acc=0.95 ---294232.36 ms/batch
Round 6 | epoch=17, train costTime=315.90173387527466, | total_activation_sum=33696777
Round 6 | epoch=18, current m_iou=0.6975649705200921
Round 6 | Best m_IoU is: 0.698, OA is: 0.922 | val costTime=146.010596036911
Round 6 | ****EPOCH 18****
Step 00092050 L_out=0.257 Acc=0.96 --- 1855.82 ms/batch
Step 00092100 L_out=0.239 Acc=0.97 ---25355.81 ms/batch
Step 00092150 L_out=0.324 Acc=0.95 ---48861.07 ms/batch
Step 00092200 L_out=0.179 Acc=0.96 ---72607.46 ms/batch
Step 00092250 L_out=0.073 Acc=0.98 ---96255.87 ms/batch
Step 00092300 L_out=0.403 Acc=0.92 ---119847.72 ms/batch
Step 00092350 L_out=0.252 Acc=0.95 ---143118.21 ms/batch
Step 00092400 L_out=0.119 Acc=0.97 ---166931.90 ms/batch
Step 00092450 L_out=0.062 Acc=0.98 ---190697.41 ms/batch
Step 00092500 L_out=0.283 Acc=0.93 ---214088.52 ms/batch
Step 00092550 L_out=0.358 Acc=0.91 ---237543.82 ms/batch
Step 00092600 L_out=0.651 Acc=0.88 ---261125.52 ms/batch
Step 00092650 L_out=0.221 Acc=0.96 ---284653.75 ms/batch
Step 00092700 L_out=0.336 Acc=0.91 ---308131.88 ms/batch
Round 6 | epoch=18, train costTime=314.2705569267273, | total_activation_sum=33774707
Round 6 | epoch=19, current m_iou=0.7036533013066112
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=153.40440011024475
Round 6 | ****EPOCH 19****
Step 00092750 L_out=0.114 Acc=0.96 ---17331.42 ms/batch
Step 00092800 L_out=0.396 Acc=0.96 ---40872.70 ms/batch
Step 00092850 L_out=0.189 Acc=0.96 ---64610.58 ms/batch
Step 00092900 L_out=0.249 Acc=0.95 ---88071.41 ms/batch
Step 00092950 L_out=0.125 Acc=0.98 ---111566.12 ms/batch
Step 00093000 L_out=0.241 Acc=0.93 ---135370.71 ms/batch
Step 00093050 L_out=0.127 Acc=0.96 ---159171.99 ms/batch
Step 00093100 L_out=0.123 Acc=0.98 ---182843.05 ms/batch
Step 00093150 L_out=0.095 Acc=0.97 ---206727.78 ms/batch
Step 00093200 L_out=0.149 Acc=0.92 ---230375.81 ms/batch
Step 00093250 L_out=0.296 Acc=0.95 ---254121.71 ms/batch
Step 00093300 L_out=0.198 Acc=0.96 ---277902.30 ms/batch
Step 00093350 L_out=0.257 Acc=0.95 ---301600.57 ms/batch
Round 6 | epoch=19, train costTime=315.72569489479065, | total_activation_sum=33749311
Round 6 | epoch=20, current m_iou=0.7007408664914931
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=145.9340205192566
Round 6 | ****EPOCH 20****
Step 00093400 L_out=0.298 Acc=0.96 --- 9665.62 ms/batch
Step 00093450 L_out=0.223 Acc=0.94 ---33474.39 ms/batch
Step 00093500 L_out=0.637 Acc=0.95 ---57257.61 ms/batch
Step 00093550 L_out=0.182 Acc=0.96 ---80760.11 ms/batch
Step 00093600 L_out=0.107 Acc=0.97 ---104488.85 ms/batch
Step 00093650 L_out=1.163 Acc=0.84 ---128059.85 ms/batch
Step 00093700 L_out=0.355 Acc=0.92 ---151828.08 ms/batch
Step 00093750 L_out=0.360 Acc=0.96 ---175645.19 ms/batch
Step 00093800 L_out=0.171 Acc=0.98 ---199152.89 ms/batch
Step 00093850 L_out=0.508 Acc=0.89 ---222829.87 ms/batch
Step 00093900 L_out=0.203 Acc=0.96 ---246669.66 ms/batch
Step 00093950 L_out=0.472 Acc=0.98 ---270449.65 ms/batch
Step 00094000 L_out=0.151 Acc=0.97 ---294049.50 ms/batch
Round 6 | epoch=20, train costTime=316.44315457344055, | total_activation_sum=33755443
Round 6 | epoch=21, current m_iou=0.6984209035742089
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=143.21843338012695
Round 6 | ****EPOCH 21****
Step 00094050 L_out=0.215 Acc=0.97 --- 1461.97 ms/batch
Step 00094100 L_out=0.245 Acc=0.96 ---24978.21 ms/batch
Step 00094150 L_out=0.266 Acc=0.98 ---48482.57 ms/batch
Step 00094200 L_out=0.166 Acc=0.96 ---72493.69 ms/batch
Step 00094250 L_out=0.092 Acc=0.97 ---96011.19 ms/batch
Step 00094300 L_out=0.173 Acc=0.95 ---119655.80 ms/batch
Step 00094350 L_out=0.227 Acc=0.95 ---143408.17 ms/batch
Step 00094400 L_out=0.205 Acc=0.96 ---167185.14 ms/batch
Step 00094450 L_out=0.116 Acc=0.97 ---191124.67 ms/batch
Step 00094500 L_out=0.098 Acc=0.98 ---214998.69 ms/batch
Step 00094550 L_out=0.224 Acc=0.97 ---238718.48 ms/batch
Step 00094600 L_out=0.233 Acc=0.95 ---262375.03 ms/batch
Step 00094650 L_out=0.133 Acc=0.96 ---286100.20 ms/batch
Step 00094700 L_out=0.325 Acc=0.91 ---309673.52 ms/batch
Round 6 | epoch=21, train costTime=316.37967920303345, | total_activation_sum=33611180
Round 6 | epoch=22, current m_iou=0.7014250560871489
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=143.72187161445618
Round 6 | ****EPOCH 22****
Step 00094750 L_out=0.253 Acc=0.97 ---16966.31 ms/batch
Step 00094800 L_out=0.717 Acc=0.90 ---40488.40 ms/batch
Step 00094850 L_out=0.166 Acc=0.95 ---64043.05 ms/batch
Step 00094900 L_out=0.112 Acc=0.98 ---87813.20 ms/batch
Step 00094950 L_out=0.650 Acc=0.87 ---111600.41 ms/batch
Step 00095000 L_out=0.161 Acc=0.97 ---135199.70 ms/batch
Step 00095050 L_out=0.203 Acc=0.96 ---158895.48 ms/batch
Step 00095100 L_out=0.211 Acc=0.96 ---182408.63 ms/batch
Step 00095150 L_out=0.202 Acc=0.96 ---206018.59 ms/batch
Step 00095200 L_out=0.382 Acc=0.95 ---229548.90 ms/batch
Step 00095250 L_out=0.044 Acc=0.97 ---253291.92 ms/batch
Step 00095300 L_out=0.121 Acc=0.97 ---276922.80 ms/batch
Step 00095350 L_out=0.235 Acc=0.96 ---300556.19 ms/batch
Round 6 | epoch=22, train costTime=315.28217339515686, | total_activation_sum=33904130
Round 6 | epoch=23, current m_iou=0.6988664166757974
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=142.49107766151428
Round 6 | ****EPOCH 23****
Step 00095400 L_out=0.123 Acc=0.98 --- 8893.56 ms/batch
Step 00095450 L_out=0.429 Acc=0.94 ---32731.60 ms/batch
Step 00095500 L_out=0.562 Acc=0.94 ---56407.31 ms/batch
Step 00095550 L_out=0.473 Acc=0.86 ---80122.47 ms/batch
Step 00095600 L_out=0.571 Acc=0.91 ---103749.16 ms/batch
Step 00095650 L_out=0.088 Acc=0.98 ---127494.59 ms/batch
Step 00095700 L_out=0.081 Acc=0.98 ---151212.56 ms/batch
Step 00095750 L_out=0.137 Acc=0.97 ---174821.94 ms/batch
Step 00095800 L_out=0.205 Acc=0.96 ---198866.00 ms/batch
Step 00095850 L_out=0.130 Acc=0.97 ---222690.08 ms/batch
Step 00095900 L_out=0.230 Acc=0.97 ---246590.43 ms/batch
Step 00095950 L_out=0.180 Acc=0.96 ---270318.67 ms/batch
Step 00096000 L_out=0.120 Acc=0.97 ---294044.55 ms/batch
Round 6 | epoch=23, train costTime=317.0095794200897, | total_activation_sum=33627901
Round 6 | epoch=24, current m_iou=0.702992095000039
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=143.6682949066162
Round 6 | ****EPOCH 24****
Step 00096050 L_out=0.248 Acc=0.90 ---  897.61 ms/batch
Step 00096100 L_out=0.189 Acc=0.97 ---24487.70 ms/batch
Step 00096150 L_out=0.188 Acc=0.96 ---48540.01 ms/batch
Step 00096200 L_out=0.063 Acc=0.99 ---72321.42 ms/batch
Step 00096250 L_out=0.118 Acc=0.97 ---95941.15 ms/batch
Step 00096300 L_out=0.094 Acc=0.97 ---119825.17 ms/batch
Step 00096350 L_out=0.116 Acc=0.94 ---143446.48 ms/batch
Step 00096400 L_out=0.207 Acc=0.96 ---167063.58 ms/batch
Step 00096450 L_out=0.272 Acc=0.97 ---190522.39 ms/batch
Step 00096500 L_out=0.274 Acc=0.94 ---214074.82 ms/batch
Step 00096550 L_out=0.379 Acc=0.92 ---237737.25 ms/batch
Step 00096600 L_out=0.243 Acc=0.97 ---261396.46 ms/batch
Step 00096650 L_out=0.529 Acc=0.94 ---285066.43 ms/batch
Step 00096700 L_out=0.112 Acc=0.96 ---308764.42 ms/batch
Round 6 | epoch=24, train costTime=315.9111840724945, | total_activation_sum=33762991
Round 6 | epoch=25, current m_iou=0.7031054612910044
Round 6 | Best m_IoU is: 0.704, OA is: 0.924 | val costTime=127.3880603313446
Round 6 | ****EPOCH 25****
Step 00096750 L_out=0.503 Acc=0.94 ---16732.25 ms/batch
Step 00096800 L_out=0.080 Acc=0.97 ---40397.51 ms/batch
Step 00096850 L_out=0.343 Acc=0.93 ---63965.95 ms/batch
Step 00096900 L_out=0.556 Acc=0.95 ---87696.29 ms/batch
Step 00096950 L_out=0.367 Acc=0.95 ---111342.26 ms/batch
Step 00097000 L_out=0.106 Acc=0.95 ---135168.74 ms/batch
Step 00097050 L_out=0.176 Acc=0.97 ---158827.38 ms/batch
Step 00097100 L_out=0.125 Acc=0.94 ---182460.64 ms/batch
Step 00097150 L_out=0.117 Acc=0.98 ---206065.68 ms/batch
Step 00097200 L_out=0.276 Acc=0.94 ---229812.56 ms/batch
Step 00097250 L_out=0.192 Acc=0.96 ---253522.36 ms/batch
Step 00097300 L_out=0.207 Acc=0.95 ---277016.68 ms/batch
Step 00097350 L_out=0.269 Acc=0.97 ---300535.38 ms/batch
Round 6 | epoch=25, train costTime=315.80614590644836, | total_activation_sum=33709590
Round 6 | epoch=26, current m_iou=0.7041169706912158
Round 6 | Best m_IoU is: 0.704, OA is: 0.925 | val costTime=127.19004130363464
Round 6 | ****EPOCH 26****
Step 00097400 L_out=0.157 Acc=0.96 --- 8497.53 ms/batch
Step 00097450 L_out=0.177 Acc=0.95 ---32195.00 ms/batch
Step 00097500 L_out=0.271 Acc=0.96 ---55854.59 ms/batch
Step 00097550 L_out=4.056 Acc=0.42 ---79573.62 ms/batch
Step 00097600 L_out=0.170 Acc=0.97 ---103131.57 ms/batch
Step 00097650 L_out=0.280 Acc=0.95 ---126724.55 ms/batch
Step 00097700 L_out=0.421 Acc=0.94 ---150685.01 ms/batch
Step 00097750 L_out=0.239 Acc=0.95 ---174168.94 ms/batch
Step 00097800 L_out=0.232 Acc=0.95 ---197718.59 ms/batch
Step 00097850 L_out=0.549 Acc=0.93 ---221415.94 ms/batch
Step 00097900 L_out=0.106 Acc=0.97 ---245135.53 ms/batch
Step 00097950 L_out=0.320 Acc=0.96 ---268961.09 ms/batch
Step 00098000 L_out=0.244 Acc=0.88 ---292639.50 ms/batch
Round 6 | epoch=26, train costTime=315.9334578514099, | total_activation_sum=33793622
Round 6 | epoch=27, current m_iou=0.704760875060556
Round 6 | Best m_IoU is: 0.705, OA is: 0.925 | val costTime=126.89826035499573
Round 6 | ****EPOCH 27****
Step 00098050 L_out=0.305 Acc=0.97 ---  485.72 ms/batch
Step 00098100 L_out=0.474 Acc=0.94 ---24007.99 ms/batch
Step 00098150 L_out=0.159 Acc=0.97 ---47812.69 ms/batch
Step 00098200 L_out=0.342 Acc=0.98 ---71605.70 ms/batch
Step 00098250 L_out=0.295 Acc=0.95 ---95151.60 ms/batch
Step 00098300 L_out=0.255 Acc=0.96 ---118861.63 ms/batch
Step 00098350 L_out=0.114 Acc=0.97 ---142370.73 ms/batch
Step 00098400 L_out=0.348 Acc=0.92 ---166232.61 ms/batch
Step 00098450 L_out=0.181 Acc=0.97 ---189799.68 ms/batch
Step 00098500 L_out=0.218 Acc=0.94 ---213534.83 ms/batch
Step 00098550 L_out=0.188 Acc=0.95 ---237142.69 ms/batch
Step 00098600 L_out=0.245 Acc=0.95 ---260845.86 ms/batch
Step 00098650 L_out=0.404 Acc=0.92 ---284524.90 ms/batch
Step 00098700 L_out=0.084 Acc=0.97 ---308299.98 ms/batch
Round 6 | epoch=27, train costTime=315.9174633026123, | total_activation_sum=33836762
Round 6 | epoch=28, current m_iou=0.7037042757421106
Round 6 | Best m_IoU is: 0.705, OA is: 0.925 | val costTime=126.82486867904663
Round 6 | ****EPOCH 28****
Step 00098750 L_out=0.244 Acc=0.96 ---16128.30 ms/batch
Step 00098800 L_out=0.147 Acc=0.96 ---39801.60 ms/batch
Step 00098850 L_out=0.127 Acc=0.97 ---63360.41 ms/batch
Step 00098900 L_out=0.077 Acc=0.97 ---87152.53 ms/batch
Step 00098950 L_out=0.411 Acc=0.93 ---110745.68 ms/batch
Step 00099000 L_out=0.803 Acc=0.91 ---134504.80 ms/batch
Step 00099050 L_out=0.137 Acc=0.98 ---158343.85 ms/batch
Step 00099100 L_out=0.160 Acc=0.97 ---181974.76 ms/batch
Step 00099150 L_out=0.774 Acc=0.89 ---205323.51 ms/batch
Step 00099200 L_out=0.343 Acc=0.96 ---228873.18 ms/batch
Step 00099250 L_out=0.142 Acc=0.96 ---252648.41 ms/batch
Step 00099300 L_out=0.128 Acc=0.96 ---276309.35 ms/batch
Step 00099350 L_out=0.264 Acc=0.95 ---299917.91 ms/batch
Round 6 | epoch=28, train costTime=315.47674679756165, | total_activation_sum=33603842
Round 6 | epoch=29, current m_iou=0.7023806948068445
Round 6 | Best m_IoU is: 0.705, OA is: 0.925 | val costTime=130.4393196105957
Round 6 | ****EPOCH 29****
Step 00099400 L_out=0.156 Acc=0.97 --- 8099.73 ms/batch
Step 00099450 L_out=0.143 Acc=0.96 ---31727.29 ms/batch
Step 00099500 L_out=0.242 Acc=0.95 ---55438.83 ms/batch
Step 00099550 L_out=0.224 Acc=0.97 ---79123.36 ms/batch
Step 00099600 L_out=0.174 Acc=0.97 ---102921.53 ms/batch
Step 00099650 L_out=0.264 Acc=0.95 ---126496.02 ms/batch
Step 00099700 L_out=0.185 Acc=0.97 ---150238.56 ms/batch
Step 00099750 L_out=0.369 Acc=0.92 ---173946.30 ms/batch
Step 00099800 L_out=0.303 Acc=0.96 ---197625.50 ms/batch
Step 00099850 L_out=0.097 Acc=0.97 ---221306.32 ms/batch
Step 00099900 L_out=0.196 Acc=0.97 ---245077.45 ms/batch
Step 00099950 L_out=0.310 Acc=0.95 ---268590.76 ms/batch
Step 00100000 L_out=0.298 Acc=0.98 ---292352.02 ms/batch
Step 00100050 L_out=0.155 Acc=0.97 ---316097.70 ms/batch
Round 6 | epoch=29, train costTime=316.09779930114746, | total_activation_sum=33638766
Round 6 | epoch=30, current m_iou=0.7048148438941393
Round 6 | Best m_IoU is: 0.705, OA is: 0.925 | val costTime=131.3336455821991
Round 6 | ****EPOCH 30****
Round 7 | ****EPOCH 0****
Step 00100100 L_out=2.969 Acc=0.61 ---23862.01 ms/batch
Step 00100150 L_out=1.295 Acc=0.84 ---47265.51 ms/batch
Step 00100200 L_out=0.684 Acc=0.89 ---70784.68 ms/batch
Step 00100250 L_out=0.863 Acc=0.89 ---94718.43 ms/batch
Step 00100300 L_out=0.685 Acc=0.89 ---118475.44 ms/batch
Step 00100350 L_out=1.247 Acc=0.83 ---142345.08 ms/batch
Step 00100400 L_out=0.715 Acc=0.95 ---166244.70 ms/batch
Step 00100450 L_out=0.939 Acc=0.93 ---189695.18 ms/batch
Step 00100500 L_out=0.222 Acc=0.95 ---213253.34 ms/batch
Step 00100550 L_out=0.888 Acc=0.86 ---236796.42 ms/batch
Step 00100600 L_out=0.653 Acc=0.92 ---260587.46 ms/batch
Step 00100650 L_out=0.334 Acc=0.95 ---284022.11 ms/batch
Step 00100700 L_out=0.229 Acc=0.95 ---307618.52 ms/batch
Round 7 | epoch=0, train costTime=315.4987223148346, | total_activation_sum=40978051
Round 7 | ****EPOCH 1****
Step 00100750 L_out=2.354 Acc=0.88 ---15708.92 ms/batch
Step 00100800 L_out=0.471 Acc=0.94 ---39274.49 ms/batch
Step 00100850 L_out=0.361 Acc=0.94 ---62812.40 ms/batch
Step 00100900 L_out=0.230 Acc=0.94 ---86678.40 ms/batch
Step 00100950 L_out=0.355 Acc=0.96 ---110285.25 ms/batch
Step 00101000 L_out=0.883 Acc=0.78 ---133720.50 ms/batch
Step 00101050 L_out=1.634 Acc=0.84 ---157373.92 ms/batch
Step 00101100 L_out=0.308 Acc=0.98 ---181081.85 ms/batch
Step 00101150 L_out=1.815 Acc=0.87 ---204791.47 ms/batch
Step 00101200 L_out=0.594 Acc=0.91 ---228449.17 ms/batch
Step 00101250 L_out=0.636 Acc=0.90 ---252033.42 ms/batch
Step 00101300 L_out=0.378 Acc=0.92 ---275568.20 ms/batch
Step 00101350 L_out=0.142 Acc=0.97 ---299218.75 ms/batch
Round 7 | epoch=1, train costTime=315.3373990058899, | total_activation_sum=40061851
Round 7 | ****EPOCH 2****
Step 00101400 L_out=0.347 Acc=0.95 --- 7709.68 ms/batch
Step 00101450 L_out=0.279 Acc=0.93 ---31445.04 ms/batch
Step 00101500 L_out=0.379 Acc=0.96 ---55050.33 ms/batch
Step 00101550 L_out=0.374 Acc=0.94 ---78678.10 ms/batch
Step 00101600 L_out=0.405 Acc=0.90 ---102292.60 ms/batch
Step 00101650 L_out=0.435 Acc=0.95 ---126070.51 ms/batch
Step 00101700 L_out=0.528 Acc=0.86 ---149715.60 ms/batch
Step 00101750 L_out=0.531 Acc=0.95 ---173535.00 ms/batch
Step 00101800 L_out=1.098 Acc=0.88 ---197019.74 ms/batch
Step 00101850 L_out=0.782 Acc=0.91 ---220379.10 ms/batch
Step 00101900 L_out=0.456 Acc=0.95 ---243910.64 ms/batch
Step 00101950 L_out=0.490 Acc=0.96 ---267543.61 ms/batch
Step 00102000 L_out=0.161 Acc=0.96 ---291130.20 ms/batch
Step 00102050 L_out=0.513 Acc=0.97 ---314752.00 ms/batch
Round 7 | epoch=2, train costTime=315.2535593509674, | total_activation_sum=40012848
Round 7 | ****EPOCH 3****
Step 00102100 L_out=0.163 Acc=0.97 ---23091.14 ms/batch
Round 2 | ****EPOCH 0****
Round 2 | ****EPOCH 0****
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.994 Acc=0.06 ---25736.69 ms/batch
Step 00000100 L_out=0.995 Acc=0.63 ---49860.77 ms/batch
Step 00000150 L_out=0.889 Acc=0.42 ---73673.84 ms/batch
Step 00000200 L_out=0.472 Acc=0.23 ---97311.85 ms/batch
Step 00000250 L_out=0.968 Acc=0.58 ---121043.38 ms/batch
Step 00000300 L_out=1.107 Acc=0.07 ---144762.12 ms/batch
Step 00000350 L_out=0.789 Acc=0.34 ---168890.07 ms/batch
Step 00000400 L_out=0.509 Acc=0.49 ---192818.59 ms/batch
Step 00000450 L_out=1.258 Acc=0.19 ---216675.97 ms/batch
Step 00000500 L_out=0.671 Acc=0.52 ---240265.41 ms/batch
Step 00000550 L_out=1.103 Acc=0.60 ---263949.33 ms/batch
Step 00000600 L_out=0.690 Acc=0.48 ---287602.22 ms/batch
Step 00000650 L_out=0.354 Acc=0.36 ---311552.55 ms/batch
Round 2 | epoch=0, train costTime=319.5982360839844, | total_activation_sum=14138789
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.968 Acc=0.35 ---15668.40 ms/batch
Step 00000750 L_out=0.435 Acc=0.65 ---39391.21 ms/batch
Step 00000800 L_out=0.751 Acc=0.40 ---63251.37 ms/batch
Step 00000850 L_out=0.673 Acc=0.57 ---87195.10 ms/batch
Step 00000900 L_out=0.268 Acc=0.47 ---110819.39 ms/batch
Step 00000950 L_out=1.286 Acc=0.28 ---134707.49 ms/batch
Step 00001000 L_out=1.216 Acc=0.47 ---158414.68 ms/batch
Step 00001050 L_out=0.448 Acc=0.54 ---182102.46 ms/batch
Step 00001100 L_out=0.439 Acc=0.39 ---205676.08 ms/batch
Step 00001150 L_out=0.973 Acc=0.44 ---229358.13 ms/batch
Step 00001200 L_out=0.925 Acc=0.73 ---252934.36 ms/batch
Step 00001250 L_out=0.296 Acc=0.43 ---276461.37 ms/batch
Step 00001300 L_out=0.480 Acc=0.62 ---300521.12 ms/batch
Round 2 | epoch=1, train costTime=316.63959550857544, | total_activation_sum=12506710
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.566 Acc=0.26 --- 7670.28 ms/batch
Step 00001400 L_out=0.902 Acc=0.37 ---31424.35 ms/batch
Step 00001450 L_out=0.336 Acc=0.39 ---55273.44 ms/batch
Step 00001500 L_out=0.442 Acc=0.68 ---78811.58 ms/batch
Step 00001550 L_out=0.417 Acc=0.41 ---102532.34 ms/batch
Step 00001600 L_out=0.328 Acc=0.59 ---126413.51 ms/batch
Step 00001650 L_out=0.528 Acc=0.35 ---150079.71 ms/batch
Step 00001700 L_out=0.689 Acc=0.75 ---173589.00 ms/batch
Step 00001750 L_out=0.993 Acc=0.64 ---197274.61 ms/batch
Step 00001800 L_out=0.664 Acc=0.60 ---221045.86 ms/batch
Step 00001850 L_out=0.470 Acc=0.77 ---244880.10 ms/batch
Step 00001900 L_out=0.639 Acc=0.43 ---268678.93 ms/batch
Step 00001950 L_out=0.385 Acc=0.51 ---292350.50 ms/batch
Step 00002000 L_out=0.351 Acc=0.60 ---316332.79 ms/batch
Round 2 | epoch=2, train costTime=316.79309487342834, | total_activation_sum=12647437
Round 2 | ****EPOCH 3****
Step 00002050 L_out=1.041 Acc=0.51 ---23307.13 ms/batch
Step 00002100 L_out=0.739 Acc=0.49 ---47080.26 ms/batch
Step 00002150 L_out=0.205 Acc=0.45 ---70808.74 ms/batch
Step 00002200 L_out=0.641 Acc=0.53 ---94568.48 ms/batch
Step 00002250 L_out=0.667 Acc=0.49 ---118403.73 ms/batch
Step 00002300 L_out=0.635 Acc=0.73 ---142134.28 ms/batch
Step 00002350 L_out=0.564 Acc=0.63 ---165668.44 ms/batch
Step 00002400 L_out=0.645 Acc=0.56 ---189398.60 ms/batch
Step 00002450 L_out=0.548 Acc=0.51 ---213063.78 ms/batch
Step 00002500 L_out=0.291 Acc=0.62 ---236777.08 ms/batch
Step 00002550 L_out=0.192 Acc=0.73 ---260520.86 ms/batch
Step 00002600 L_out=1.158 Acc=0.69 ---284222.34 ms/batch
Step 00002650 L_out=0.491 Acc=0.56 ---307952.86 ms/batch
Round 2 | epoch=3, train costTime=316.42343068122864, | total_activation_sum=12569030
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.362 Acc=0.74 ---15097.08 ms/batch
Step 00002750 L_out=0.968 Acc=0.75 ---38779.39 ms/batch
Step 00002800 L_out=0.263 Acc=0.77 ---62536.83 ms/batch
Step 00002850 L_out=0.686 Acc=0.44 ---86061.80 ms/batch
Step 00002900 L_out=0.393 Acc=0.73 ---109577.26 ms/batch
Step 00002950 L_out=0.326 Acc=0.25 ---133073.17 ms/batch
Step 00003000 L_out=0.541 Acc=0.69 ---156778.76 ms/batch
Step 00003050 L_out=0.883 Acc=0.67 ---180498.69 ms/batch
Step 00003100 L_out=0.380 Acc=0.46 ---204213.65 ms/batch
Step 00003150 L_out=0.280 Acc=0.63 ---227799.51 ms/batch
Step 00003200 L_out=0.907 Acc=0.61 ---251645.49 ms/batch
Step 00003250 L_out=0.270 Acc=0.61 ---275447.98 ms/batch
Step 00003300 L_out=0.470 Acc=0.75 ---299132.80 ms/batch
Round 2 | epoch=4, train costTime=315.8786664009094, | total_activation_sum=12653442
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.306 Acc=0.75 --- 7169.10 ms/batch
Step 00003400 L_out=0.415 Acc=0.75 ---30990.92 ms/batch
Step 00003450 L_out=0.451 Acc=0.62 ---54749.42 ms/batch
Step 00003500 L_out=0.724 Acc=0.63 ---78547.87 ms/batch
Step 00003550 L_out=0.392 Acc=0.77 ---102359.59 ms/batch
Step 00003600 L_out=0.339 Acc=0.66 ---126112.18 ms/batch
Step 00003650 L_out=1.487 Acc=0.48 ---149898.83 ms/batch
Step 00003700 L_out=0.346 Acc=0.67 ---173751.60 ms/batch
Step 00003750 L_out=0.139 Acc=0.72 ---197512.21 ms/batch
Step 00003800 L_out=0.364 Acc=0.73 ---221357.18 ms/batch
Step 00003850 L_out=0.354 Acc=0.54 ---244928.14 ms/batch
Step 00003900 L_out=0.398 Acc=0.82 ---268797.59 ms/batch
Step 00003950 L_out=0.301 Acc=0.52 ---292573.70 ms/batch
Step 00004000 L_out=0.360 Acc=0.75 ---316225.22 ms/batch
Round 2 | epoch=5, train costTime=317.17762756347656, | total_activation_sum=12789047
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.083 Acc=0.58 ---22731.86 ms/batch
Step 00004100 L_out=0.133 Acc=0.74 ---46369.65 ms/batch
Step 00004150 L_out=1.031 Acc=0.51 ---70114.39 ms/batch
Step 00004200 L_out=0.508 Acc=0.64 ---93714.49 ms/batch
Step 00004250 L_out=0.102 Acc=0.81 ---117492.99 ms/batch
Step 00004300 L_out=0.417 Acc=0.72 ---141192.47 ms/batch
Step 00004350 L_out=0.671 Acc=0.73 ---165018.20 ms/batch
Step 00004400 L_out=0.249 Acc=0.80 ---188671.78 ms/batch
Step 00004450 L_out=0.474 Acc=0.62 ---212446.90 ms/batch
Step 00004500 L_out=0.839 Acc=0.75 ---236208.59 ms/batch
Step 00004550 L_out=0.354 Acc=0.87 ---259887.26 ms/batch
Step 00004600 L_out=0.380 Acc=0.77 ---283684.79 ms/batch
Step 00004650 L_out=0.290 Acc=0.59 ---307383.55 ms/batch
Round 2 | epoch=6, train costTime=316.43940114974976, | total_activation_sum=12659163
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.472 Acc=0.69 ---14636.29 ms/batch
Step 00004750 L_out=0.604 Acc=0.52 ---38152.38 ms/batch
Step 00004800 L_out=0.405 Acc=0.67 ---61902.40 ms/batch
Step 00004850 L_out=0.331 Acc=0.72 ---85743.86 ms/batch
Step 00004900 L_out=0.426 Acc=0.66 ---109534.68 ms/batch
Step 00004950 L_out=0.152 Acc=0.84 ---133262.04 ms/batch
Step 00005000 L_out=0.451 Acc=0.64 ---156995.33 ms/batch
Step 00005050 L_out=0.473 Acc=0.83 ---180676.96 ms/batch
Step 00005100 L_out=0.082 Acc=0.82 ---204583.65 ms/batch
Step 00005150 L_out=0.339 Acc=0.72 ---228363.27 ms/batch
Step 00005200 L_out=0.541 Acc=0.62 ---252181.73 ms/batch
Step 00005250 L_out=1.286 Acc=0.73 ---275799.12 ms/batch
Step 00005300 L_out=0.145 Acc=0.86 ---299307.60 ms/batch
Round 2 | epoch=7, train costTime=316.262482881546, | total_activation_sum=12747700
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.121 Acc=0.79 --- 6593.86 ms/batch
Step 00005400 L_out=0.207 Acc=0.82 ---30304.70 ms/batch
Step 00005450 L_out=0.182 Acc=0.88 ---53835.80 ms/batch
Step 00005500 L_out=0.121 Acc=0.90 ---77595.60 ms/batch
Step 00005550 L_out=0.208 Acc=0.81 ---101258.26 ms/batch
Step 00005600 L_out=0.191 Acc=0.77 ---125110.68 ms/batch
Step 00005650 L_out=0.417 Acc=0.77 ---148655.42 ms/batch
Step 00005700 L_out=0.363 Acc=0.78 ---172654.87 ms/batch
Step 00005750 L_out=0.187 Acc=0.78 ---196598.44 ms/batch
Step 00005800 L_out=0.129 Acc=0.78 ---220465.32 ms/batch
Step 00005850 L_out=0.497 Acc=0.78 ---244141.89 ms/batch
Step 00005900 L_out=0.181 Acc=0.76 ---267851.21 ms/batch
Step 00005950 L_out=0.210 Acc=0.86 ---291622.58 ms/batch
Step 00006000 L_out=0.135 Acc=0.79 ---315400.03 ms/batch
Round 2 | epoch=8, train costTime=316.7663950920105, | total_activation_sum=12578783
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.291 Acc=0.71 ---22087.41 ms/batch
Step 00006100 L_out=0.184 Acc=0.85 ---45725.50 ms/batch
Step 00006150 L_out=0.239 Acc=0.81 ---69559.01 ms/batch
Step 00006200 L_out=0.212 Acc=0.92 ---93363.48 ms/batch
Step 00006250 L_out=0.058 Acc=0.79 ---117058.18 ms/batch
Step 00006300 L_out=0.135 Acc=0.71 ---140524.71 ms/batch
Step 00006350 L_out=0.160 Acc=0.71 ---164200.22 ms/batch
Step 00006400 L_out=0.232 Acc=0.88 ---188062.62 ms/batch
Step 00006450 L_out=0.326 Acc=0.73 ---211706.32 ms/batch
Step 00006500 L_out=0.251 Acc=0.80 ---235473.58 ms/batch
Step 00006550 L_out=0.297 Acc=0.89 ---259070.55 ms/batch
Step 00006600 L_out=0.573 Acc=0.83 ---282912.50 ms/batch
Step 00006650 L_out=0.277 Acc=0.81 ---306805.88 ms/batch
Round 2 | epoch=9, train costTime=316.37564396858215, | total_activation_sum=12574375
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.228 Acc=0.85 ---14116.75 ms/batch
Step 00006750 L_out=0.267 Acc=0.83 ---37963.23 ms/batch
Step 00006800 L_out=0.128 Acc=0.91 ---61536.60 ms/batch
Step 00006850 L_out=0.275 Acc=0.83 ---85022.10 ms/batch
Step 00006900 L_out=0.332 Acc=0.89 ---108878.34 ms/batch
Step 00006950 L_out=0.258 Acc=0.89 ---132456.76 ms/batch
Step 00007000 L_out=0.177 Acc=0.74 ---156052.36 ms/batch
Step 00007050 L_out=0.394 Acc=0.88 ---179867.61 ms/batch
Step 00007100 L_out=0.200 Acc=0.85 ---203466.14 ms/batch
Step 00007150 L_out=0.134 Acc=0.84 ---227024.34 ms/batch
Step 00007200 L_out=0.237 Acc=0.84 ---250853.63 ms/batch
Step 00007250 L_out=0.552 Acc=0.81 ---274564.53 ms/batch
Step 00007300 L_out=0.125 Acc=0.90 ---298407.32 ms/batch
Round 2 | epoch=10, train costTime=315.9539284706116, | total_activation_sum=12731299
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.344 Acc=0.69 --- 6144.54 ms/batch
Step 00007400 L_out=0.386 Acc=0.81 ---29807.45 ms/batch
Step 00007450 L_out=0.036 Acc=0.84 ---53509.56 ms/batch
Step 00007500 L_out=0.293 Acc=0.64 ---77197.78 ms/batch
Step 00007550 L_out=0.410 Acc=0.76 ---101153.08 ms/batch
Step 00007600 L_out=0.433 Acc=0.78 ---124554.57 ms/batch
Step 00007650 L_out=0.711 Acc=0.79 ---148435.33 ms/batch
Step 00007700 L_out=0.320 Acc=0.82 ---172216.92 ms/batch
Step 00007750 L_out=0.106 Acc=0.87 ---195989.59 ms/batch
Step 00007800 L_out=0.313 Acc=0.84 ---219626.43 ms/batch
Step 00007850 L_out=0.549 Acc=0.73 ---243278.44 ms/batch
Step 00007900 L_out=0.156 Acc=0.80 ---267135.30 ms/batch
Step 00007950 L_out=0.371 Acc=0.87 ---290870.54 ms/batch
Step 00008000 L_out=0.126 Acc=0.91 ---314761.62 ms/batch
Round 2 | epoch=11, train costTime=316.7441110610962, | total_activation_sum=12793213
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.389 Acc=0.81 ---21896.17 ms/batch
Step 00008100 L_out=0.095 Acc=0.87 ---45629.97 ms/batch
Step 00008150 L_out=0.309 Acc=0.83 ---69502.53 ms/batch
Step 00008200 L_out=0.363 Acc=0.77 ---93169.31 ms/batch
Step 00008250 L_out=0.108 Acc=0.72 ---116718.71 ms/batch
Step 00008300 L_out=0.148 Acc=0.83 ---140524.72 ms/batch
Step 00008350 L_out=0.100 Acc=0.89 ---164125.82 ms/batch
Step 00008400 L_out=0.108 Acc=0.88 ---187965.98 ms/batch
Step 00008450 L_out=0.225 Acc=0.77 ---211639.57 ms/batch
Step 00008500 L_out=0.573 Acc=0.88 ---235393.01 ms/batch
Step 00008550 L_out=0.543 Acc=0.88 ---259392.19 ms/batch
Step 00008600 L_out=0.099 Acc=0.90 ---282941.75 ms/batch
Step 00008650 L_out=0.265 Acc=0.75 ---306460.58 ms/batch
Round 2 | epoch=12, train costTime=316.4484980106354, | total_activation_sum=12644404
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.121 Acc=0.85 ---13677.74 ms/batch
Step 00008750 L_out=0.377 Acc=0.80 ---37234.21 ms/batch
Step 00008800 L_out=0.368 Acc=0.80 ---61098.90 ms/batch
Step 00008850 L_out=0.071 Acc=0.89 ---84636.74 ms/batch
Step 00008900 L_out=0.408 Acc=0.79 ---108416.48 ms/batch
Step 00008950 L_out=0.151 Acc=0.86 ---132026.01 ms/batch
Step 00009000 L_out=0.248 Acc=0.68 ---155715.17 ms/batch
Step 00009050 L_out=0.342 Acc=0.61 ---179409.77 ms/batch
Step 00009100 L_out=0.313 Acc=0.89 ---203319.25 ms/batch
Step 00009150 L_out=0.076 Acc=0.89 ---227061.29 ms/batch
Step 00009200 L_out=0.132 Acc=0.82 ---250980.54 ms/batch
Step 00009250 L_out=0.213 Acc=0.85 ---274794.74 ms/batch
Step 00009300 L_out=0.511 Acc=0.75 ---298546.81 ms/batch
Round 2 | epoch=13, train costTime=316.57973408699036, | total_activation_sum=12736286
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.166 Acc=0.85 --- 5657.52 ms/batch
Step 00009400 L_out=0.278 Acc=0.83 ---29334.61 ms/batch
Step 00009450 L_out=0.113 Acc=0.89 ---52984.11 ms/batch
Step 00009500 L_out=0.055 Acc=0.88 ---76809.18 ms/batch
Step 00009550 L_out=0.105 Acc=0.77 ---100305.31 ms/batch
Step 00009600 L_out=0.159 Acc=0.84 ---124041.88 ms/batch
Step 00009650 L_out=0.242 Acc=0.89 ---147735.23 ms/batch
Step 00009700 L_out=0.215 Acc=0.81 ---171478.65 ms/batch
Step 00009750 L_out=0.099 Acc=0.82 ---195183.23 ms/batch
Step 00009800 L_out=0.157 Acc=0.75 ---219039.41 ms/batch
Step 00009850 L_out=0.267 Acc=0.82 ---242764.81 ms/batch
Step 00009900 L_out=0.111 Acc=0.83 ---266528.28 ms/batch
Step 00009950 L_out=0.049 Acc=0.84 ---290162.97 ms/batch
Step 00010000 L_out=0.186 Acc=0.88 ---313904.06 ms/batch
Round 2 | epoch=14, train costTime=316.20400953292847, | total_activation_sum=12686156
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.186 Acc=0.75 ---21397.88 ms/batch
Step 00010100 L_out=0.242 Acc=0.85 ---45339.34 ms/batch
Step 00010150 L_out=0.432 Acc=0.84 ---69172.39 ms/batch
Step 00010200 L_out=0.170 Acc=0.79 ---92875.15 ms/batch
Step 00010250 L_out=0.179 Acc=0.87 ---116619.83 ms/batch
Step 00010300 L_out=0.208 Acc=0.90 ---140244.96 ms/batch
Step 00010350 L_out=0.119 Acc=0.89 ---163982.27 ms/batch
Step 00010400 L_out=0.057 Acc=0.83 ---187604.01 ms/batch
Step 00010450 L_out=0.152 Acc=0.90 ---211276.13 ms/batch
Step 00010500 L_out=0.211 Acc=0.87 ---234999.42 ms/batch
Step 00010550 L_out=0.210 Acc=0.89 ---258711.21 ms/batch
Step 00010600 L_out=0.045 Acc=0.87 ---282369.20 ms/batch
Step 00010650 L_out=0.147 Acc=0.83 ---305833.28 ms/batch
Round 2 | epoch=15, train costTime=316.42377829551697, | total_activation_sum=12689258
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.040 Acc=0.68 ---13198.88 ms/batch
Step 00010750 L_out=0.414 Acc=0.81 ---36931.94 ms/batch
Step 00010800 L_out=0.075 Acc=0.93 ---60708.56 ms/batch
Step 00010850 L_out=0.985 Acc=0.70 ---84410.71 ms/batch
Step 00010900 L_out=0.341 Acc=0.91 ---108208.55 ms/batch
Step 00010950 L_out=0.401 Acc=0.88 ---131798.09 ms/batch
Step 00011000 L_out=0.451 Acc=0.86 ---155508.28 ms/batch
Step 00011050 L_out=0.140 Acc=0.91 ---179129.21 ms/batch
Step 00011100 L_out=0.265 Acc=0.93 ---202872.10 ms/batch
Step 00011150 L_out=0.095 Acc=0.91 ---226586.38 ms/batch
Step 00011200 L_out=0.120 Acc=0.91 ---250451.01 ms/batch
Step 00011250 L_out=0.142 Acc=0.46 ---274167.00 ms/batch
Step 00011300 L_out=0.119 Acc=0.91 ---297588.52 ms/batch
Round 2 | epoch=16, train costTime=316.1203317642212, | total_activation_sum=12679603
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.304 Acc=0.70 --- 5214.13 ms/batch
Step 00011400 L_out=0.116 Acc=0.80 ---28960.35 ms/batch
Step 00011450 L_out=0.101 Acc=0.78 ---52617.32 ms/batch
Step 00011500 L_out=0.240 Acc=0.81 ---76371.22 ms/batch
Step 00011550 L_out=0.106 Acc=0.75 ---99964.04 ms/batch
Step 00011600 L_out=0.157 Acc=0.88 ---123547.98 ms/batch
Step 00011650 L_out=0.463 Acc=0.78 ---147100.87 ms/batch
Step 00011700 L_out=0.206 Acc=0.91 ---170893.78 ms/batch
Step 00011750 L_out=0.090 Acc=0.92 ---194700.55 ms/batch
Step 00011800 L_out=0.032 Acc=0.92 ---218476.42 ms/batch
Step 00011850 L_out=0.037 Acc=0.91 ---242307.66 ms/batch
Step 00011900 L_out=0.093 Acc=0.86 ---266015.25 ms/batch
Step 00011950 L_out=0.203 Acc=0.90 ---289684.11 ms/batch
Step 00012000 L_out=0.150 Acc=0.85 ---313326.28 ms/batch
Round 2 | epoch=17, train costTime=316.19164967536926, | total_activation_sum=12615066
Round 2 | epoch=18, current m_iou=0.6184190662715744
Round 2 | Best m_IoU is: 0.618, OA is: 0.877 | val costTime=120.49667453765869
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.459 Acc=0.90 ---20699.46 ms/batch
Step 00012100 L_out=0.132 Acc=0.83 ---44440.62 ms/batch
Step 00012150 L_out=0.270 Acc=0.82 ---68177.30 ms/batch
Step 00012200 L_out=0.156 Acc=0.89 ---91889.41 ms/batch
Step 00012250 L_out=0.372 Acc=0.78 ---115610.32 ms/batch
Step 00012300 L_out=0.036 Acc=0.90 ---139241.11 ms/batch
Step 00012350 L_out=0.119 Acc=0.80 ---163160.99 ms/batch
Step 00012400 L_out=0.063 Acc=0.81 ---186953.38 ms/batch
Step 00012450 L_out=0.154 Acc=0.85 ---210929.97 ms/batch
Step 00012500 L_out=3.178 Acc=0.33 ---234521.23 ms/batch
Step 00012550 L_out=0.161 Acc=0.76 ---258219.14 ms/batch
Step 00012600 L_out=0.508 Acc=0.64 ---281771.07 ms/batch
Step 00012650 L_out=0.058 Acc=0.81 ---305480.48 ms/batch
Round 2 | epoch=18, train costTime=316.41407346725464, | total_activation_sum=12751605
Round 2 | epoch=19, current m_iou=0.6055027286880759
Round 2 | Best m_IoU is: 0.618, OA is: 0.877 | val costTime=117.38782477378845
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.134 Acc=0.91 ---12833.16 ms/batch
Step 00012750 L_out=0.095 Acc=0.91 ---36461.60 ms/batch
Step 00012800 L_out=0.169 Acc=0.87 ---60229.49 ms/batch
Step 00012850 L_out=0.046 Acc=0.78 ---84166.02 ms/batch
Step 00012900 L_out=0.164 Acc=0.68 ---107899.43 ms/batch
Step 00012950 L_out=0.058 Acc=0.85 ---131442.86 ms/batch
Step 00013000 L_out=0.143 Acc=0.79 ---155164.80 ms/batch
Step 00013050 L_out=0.075 Acc=0.83 ---178858.10 ms/batch
Step 00013100 L_out=0.168 Acc=0.86 ---202717.35 ms/batch
Step 00013150 L_out=0.126 Acc=0.88 ---226297.73 ms/batch
Step 00013200 L_out=0.021 Acc=0.91 ---250133.30 ms/batch
Step 00013250 L_out=0.169 Acc=0.88 ---273795.95 ms/batch
Step 00013300 L_out=0.082 Acc=0.85 ---297464.09 ms/batch
Round 2 | epoch=19, train costTime=316.59981298446655, | total_activation_sum=12813943
Round 2 | epoch=20, current m_iou=0.6151718392388789
Round 2 | Best m_IoU is: 0.618, OA is: 0.877 | val costTime=117.54501581192017
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.149 Acc=0.90 --- 4806.98 ms/batch
Step 00013400 L_out=0.382 Acc=0.90 ---28549.12 ms/batch
Step 00013450 L_out=0.009 Acc=0.88 ---52180.00 ms/batch
Step 00013500 L_out=0.054 Acc=0.91 ---75895.64 ms/batch
Step 00013550 L_out=0.102 Acc=0.92 ---99613.51 ms/batch
Step 00013600 L_out=0.269 Acc=0.83 ---123414.54 ms/batch
Step 00013650 L_out=0.179 Acc=0.90 ---147169.95 ms/batch
Step 00013700 L_out=0.192 Acc=0.88 ---170928.09 ms/batch
Step 00013750 L_out=0.156 Acc=0.87 ---194568.73 ms/batch
Step 00013800 L_out=0.174 Acc=0.84 ---218356.00 ms/batch
Step 00013850 L_out=0.324 Acc=0.83 ---242174.16 ms/batch
Step 00013900 L_out=0.275 Acc=0.65 ---266014.99 ms/batch
Step 00013950 L_out=0.162 Acc=0.84 ---289735.51 ms/batch
Step 00014000 L_out=0.086 Acc=0.84 ---313491.50 ms/batch
Round 2 | epoch=20, train costTime=316.8306064605713, | total_activation_sum=12639053
Round 2 | epoch=21, current m_iou=0.6228586738850518
Round 2 | Best m_IoU is: 0.623, OA is: 0.885 | val costTime=117.0125458240509
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.064 Acc=0.86 ---20460.89 ms/batch
Step 00014100 L_out=0.120 Acc=0.94 ---43969.08 ms/batch
Step 00014150 L_out=0.138 Acc=0.87 ---67788.80 ms/batch
Step 00014200 L_out=0.146 Acc=0.91 ---91557.86 ms/batch
Step 00014250 L_out=0.113 Acc=0.83 ---115341.40 ms/batch
Step 00014300 L_out=0.317 Acc=0.76 ---138874.58 ms/batch
Step 00014350 L_out=0.207 Acc=0.88 ---162403.63 ms/batch
Step 00014400 L_out=0.323 Acc=0.86 ---186170.49 ms/batch
Step 00014450 L_out=0.105 Acc=0.85 ---209683.82 ms/batch
Step 00014500 L_out=0.296 Acc=0.74 ---233367.80 ms/batch
Step 00014550 L_out=0.257 Acc=0.63 ---257175.41 ms/batch
Step 00014600 L_out=0.236 Acc=0.89 ---280766.03 ms/batch
Step 00014650 L_out=0.126 Acc=0.88 ---304446.64 ms/batch
Round 2 | epoch=21, train costTime=315.9111452102661, | total_activation_sum=12668298
Round 2 | epoch=22, current m_iou=0.6251766505922896
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.23622226715088
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.057 Acc=0.85 ---12243.92 ms/batch
Step 00014750 L_out=0.100 Acc=0.84 ---35954.51 ms/batch
Step 00014800 L_out=0.132 Acc=0.81 ---59645.66 ms/batch
Step 00014850 L_out=0.199 Acc=0.89 ---83526.82 ms/batch
Step 00014900 L_out=0.818 Acc=0.64 ---107226.08 ms/batch
Step 00014950 L_out=0.047 Acc=0.91 ---130788.06 ms/batch
Step 00015000 L_out=0.085 Acc=0.92 ---154491.48 ms/batch
Step 00015050 L_out=0.059 Acc=0.94 ---178347.85 ms/batch
Step 00015100 L_out=0.141 Acc=0.85 ---201753.34 ms/batch
Step 00015150 L_out=0.082 Acc=0.83 ---225656.53 ms/batch
Step 00015200 L_out=0.208 Acc=0.91 ---249295.93 ms/batch
Step 00015250 L_out=0.205 Acc=0.90 ---272912.83 ms/batch
Step 00015300 L_out=0.213 Acc=0.86 ---296543.26 ms/batch
Round 2 | epoch=22, train costTime=315.9671382904053, | total_activation_sum=12662922
Round 2 | epoch=23, current m_iou=0.6200089911019631
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.21526527404785
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.129 Acc=0.69 --- 4231.60 ms/batch
Step 00015400 L_out=0.106 Acc=0.85 ---27775.73 ms/batch
Step 00015450 L_out=0.173 Acc=0.61 ---51462.71 ms/batch
Step 00015500 L_out=0.189 Acc=0.88 ---75212.49 ms/batch
Step 00015550 L_out=0.387 Acc=0.82 ---98936.55 ms/batch
Step 00015600 L_out=0.090 Acc=0.94 ---122488.24 ms/batch
Step 00015650 L_out=0.148 Acc=0.87 ---146177.96 ms/batch
Step 00015700 L_out=0.158 Acc=0.93 ---169957.15 ms/batch
Step 00015750 L_out=0.055 Acc=0.90 ---193550.48 ms/batch
Step 00015800 L_out=0.087 Acc=0.92 ---217075.78 ms/batch
Step 00015850 L_out=0.070 Acc=0.91 ---240684.15 ms/batch
Step 00015900 L_out=0.044 Acc=0.87 ---264285.73 ms/batch
Step 00015950 L_out=0.084 Acc=0.89 ---287936.84 ms/batch
Step 00016000 L_out=0.076 Acc=0.89 ---311642.67 ms/batch
Round 2 | epoch=23, train costTime=315.39909625053406, | total_activation_sum=12777732
Round 2 | epoch=24, current m_iou=0.5997984864765287
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=116.89041423797607
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.146 Acc=0.75 ---19714.44 ms/batch
Step 00016100 L_out=0.101 Acc=0.91 ---43459.48 ms/batch
Step 00016150 L_out=0.188 Acc=0.84 ---67165.17 ms/batch
Step 00016200 L_out=0.225 Acc=0.89 ---90768.41 ms/batch
Step 00016250 L_out=0.152 Acc=0.70 ---114284.09 ms/batch
Step 00016300 L_out=0.165 Acc=0.91 ---137935.57 ms/batch
Step 00016350 L_out=0.071 Acc=0.95 ---161636.47 ms/batch
Step 00016400 L_out=0.046 Acc=0.86 ---185485.00 ms/batch
Step 00016450 L_out=0.059 Acc=0.89 ---208944.08 ms/batch
Step 00016500 L_out=0.081 Acc=0.89 ---232590.10 ms/batch
Step 00016550 L_out=0.315 Acc=0.86 ---256417.37 ms/batch
Step 00016600 L_out=0.138 Acc=0.87 ---279983.47 ms/batch
Step 00016650 L_out=0.078 Acc=0.95 ---303614.13 ms/batch
Round 2 | epoch=24, train costTime=315.5113139152527, | total_activation_sum=12593291
Round 2 | epoch=25, current m_iou=0.6171464185882258
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.03217601776123
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.144 Acc=0.80 ---11917.12 ms/batch
Step 00016750 L_out=0.049 Acc=0.91 ---35607.37 ms/batch
Step 00016800 L_out=0.072 Acc=0.93 ---59135.01 ms/batch
Step 00016850 L_out=0.059 Acc=0.91 ---82763.10 ms/batch
Step 00016900 L_out=0.188 Acc=0.85 ---106558.50 ms/batch
Step 00016950 L_out=0.167 Acc=0.84 ---130127.80 ms/batch
Step 00017000 L_out=0.033 Acc=0.95 ---153622.06 ms/batch
Step 00017050 L_out=0.163 Acc=0.90 ---177071.99 ms/batch
Step 00017100 L_out=0.188 Acc=0.88 ---200746.77 ms/batch
Step 00017150 L_out=0.403 Acc=0.85 ---224508.19 ms/batch
Step 00017200 L_out=0.134 Acc=0.88 ---248001.61 ms/batch
Step 00017250 L_out=0.085 Acc=0.92 ---271758.16 ms/batch
Step 00017300 L_out=0.205 Acc=0.90 ---295431.44 ms/batch
Round 2 | epoch=25, train costTime=315.18269777297974, | total_activation_sum=12680380
Round 2 | epoch=26, current m_iou=0.6116851212216196
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.2992172241211
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.229 Acc=0.88 --- 3733.93 ms/batch
Step 00017400 L_out=0.036 Acc=0.86 ---27389.39 ms/batch
Step 00017450 L_out=0.282 Acc=0.81 ---51441.68 ms/batch
Step 00017500 L_out=0.075 Acc=0.93 ---74906.17 ms/batch
Step 00017550 L_out=0.812 Acc=0.90 ---98541.08 ms/batch
Step 00017600 L_out=0.040 Acc=0.79 ---122042.04 ms/batch
Step 00017650 L_out=0.079 Acc=0.95 ---145840.23 ms/batch
Step 00017700 L_out=0.307 Acc=0.88 ---169449.83 ms/batch
Step 00017750 L_out=0.132 Acc=0.89 ---193105.22 ms/batch
Step 00017800 L_out=0.060 Acc=0.85 ---216662.05 ms/batch
Step 00017850 L_out=0.043 Acc=0.92 ---240383.26 ms/batch
Step 00017900 L_out=0.204 Acc=0.81 ---264032.49 ms/batch
Step 00017950 L_out=0.133 Acc=0.87 ---287662.91 ms/batch
Step 00018000 L_out=0.079 Acc=0.83 ---311329.97 ms/batch
Round 2 | epoch=26, train costTime=315.5767722129822, | total_activation_sum=12742027
Round 2 | epoch=27, current m_iou=0.6061186139031646
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=116.96205425262451
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.096 Acc=0.91 ---19158.12 ms/batch
Step 00018100 L_out=0.152 Acc=0.88 ---43085.98 ms/batch
Step 00018150 L_out=0.759 Acc=0.75 ---66796.20 ms/batch
Step 00018200 L_out=0.026 Acc=0.93 ---90709.48 ms/batch
Step 00018250 L_out=0.163 Acc=0.83 ---114406.44 ms/batch
Step 00018300 L_out=0.428 Acc=0.82 ---138083.34 ms/batch
Step 00018350 L_out=0.107 Acc=0.83 ---161671.33 ms/batch
Step 00018400 L_out=0.130 Acc=0.92 ---185361.46 ms/batch
Step 00018450 L_out=0.051 Acc=0.86 ---209126.85 ms/batch
Step 00018500 L_out=0.036 Acc=0.92 ---232692.46 ms/batch
Step 00018550 L_out=0.202 Acc=0.90 ---256467.04 ms/batch
Step 00018600 L_out=0.141 Acc=0.86 ---280219.92 ms/batch
Step 00018650 L_out=0.076 Acc=0.90 ---303942.68 ms/batch
Round 2 | epoch=27, train costTime=316.3996992111206, | total_activation_sum=12700664
Round 2 | epoch=28, current m_iou=0.6021670830257085
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.41497349739075
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.079 Acc=0.93 ---11335.14 ms/batch
Step 00018750 L_out=0.036 Acc=0.91 ---35009.02 ms/batch
Step 00018800 L_out=0.159 Acc=0.84 ---58593.93 ms/batch
Step 00018850 L_out=0.107 Acc=0.84 ---82449.20 ms/batch
Step 00018900 L_out=0.133 Acc=0.90 ---106053.41 ms/batch
Step 00018950 L_out=0.078 Acc=0.86 ---129650.29 ms/batch
Step 00019000 L_out=0.147 Acc=0.94 ---153382.77 ms/batch
Step 00019050 L_out=0.069 Acc=0.92 ---176941.91 ms/batch
Step 00019100 L_out=0.120 Acc=0.87 ---200445.47 ms/batch
Step 00019150 L_out=0.021 Acc=0.85 ---224148.67 ms/batch
Step 00019200 L_out=0.155 Acc=0.80 ---248120.93 ms/batch
Step 00019250 L_out=0.068 Acc=0.91 ---271943.38 ms/batch
Step 00019300 L_out=0.149 Acc=0.90 ---295624.54 ms/batch
Round 2 | epoch=28, train costTime=316.05856919288635, | total_activation_sum=12706049
Round 2 | epoch=29, current m_iou=0.6083515745828192
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.5589747428894
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.092 Acc=0.90 --- 3288.96 ms/batch
Step 00019400 L_out=0.138 Acc=0.89 ---26778.23 ms/batch
Step 00019450 L_out=0.063 Acc=0.95 ---50523.53 ms/batch
Step 00019500 L_out=0.144 Acc=0.92 ---74290.98 ms/batch
Step 00019550 L_out=0.082 Acc=0.79 ---97911.84 ms/batch
Step 00019600 L_out=0.078 Acc=0.88 ---121540.82 ms/batch
Step 00019650 L_out=0.103 Acc=0.87 ---145257.78 ms/batch
Step 00019700 L_out=0.035 Acc=0.90 ---168945.44 ms/batch
Step 00019750 L_out=0.258 Acc=0.76 ---192344.49 ms/batch
Step 00019800 L_out=0.163 Acc=0.87 ---216244.69 ms/batch
Step 00019850 L_out=0.027 Acc=0.92 ---239966.82 ms/batch
Step 00019900 L_out=0.128 Acc=0.89 ---263471.23 ms/batch
Step 00019950 L_out=0.043 Acc=0.91 ---287191.57 ms/batch
Step 00020000 L_out=0.079 Acc=0.91 ---310675.80 ms/batch
Round 2 | epoch=29, train costTime=315.4384665489197, | total_activation_sum=12757292
Round 2 | epoch=30, current m_iou=0.6208452207651023
Round 2 | Best m_IoU is: 0.625, OA is: 0.887 | val costTime=117.15951251983643
Round 2 | ****EPOCH 30****
Round 2 | ****EPOCH 0****
Step 00000050 L_out=1.133 Acc=0.33 ---25506.79 ms/batch
Step 00000100 L_out=0.664 Acc=0.71 ---49070.42 ms/batch
Step 00000150 L_out=0.934 Acc=0.45 ---72440.47 ms/batch
Step 00000200 L_out=0.247 Acc=0.38 ---96173.86 ms/batch
Step 00000250 L_out=1.029 Acc=0.32 ---119430.29 ms/batch
Step 00000300 L_out=0.633 Acc=0.43 ---142911.18 ms/batch
Step 00000350 L_out=0.390 Acc=0.51 ---166701.67 ms/batch
Step 00000400 L_out=0.754 Acc=0.19 ---190145.46 ms/batch
Step 00000450 L_out=0.395 Acc=0.21 ---213560.65 ms/batch
Step 00000500 L_out=0.417 Acc=0.57 ---236842.06 ms/batch
Step 00000550 L_out=2.088 Acc=0.48 ---260255.07 ms/batch
Step 00000600 L_out=1.404 Acc=0.34 ---283932.11 ms/batch
Step 00000650 L_out=0.757 Acc=0.55 ---307531.61 ms/batch
Round 2 | epoch=0, train costTime=315.479421377182, | total_activation_sum=14102368
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.491 Acc=0.46 ---15426.67 ms/batch
Step 00000750 L_out=0.441 Acc=0.50 ---38615.44 ms/batch
Step 00000800 L_out=0.523 Acc=0.51 ---62118.43 ms/batch
Step 00000850 L_out=0.505 Acc=0.46 ---85588.79 ms/batch
Step 00000900 L_out=0.572 Acc=0.65 ---108949.83 ms/batch
Step 00000950 L_out=1.229 Acc=0.40 ---132324.45 ms/batch
Step 00001000 L_out=0.610 Acc=0.49 ---155624.55 ms/batch
Step 00001050 L_out=0.860 Acc=0.54 ---178969.41 ms/batch
Step 00001100 L_out=0.727 Acc=0.43 ---202429.97 ms/batch
Step 00001150 L_out=0.875 Acc=0.50 ---225831.42 ms/batch
Step 00001200 L_out=0.999 Acc=0.33 ---249200.28 ms/batch
Step 00001250 L_out=0.728 Acc=0.55 ---272712.65 ms/batch
Step 00001300 L_out=1.704 Acc=0.42 ---296170.29 ms/batch
Round 2 | epoch=1, train costTime=311.93941950798035, | total_activation_sum=12359678
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.361 Acc=0.50 --- 7450.57 ms/batch
Step 00001400 L_out=0.909 Acc=0.63 ---30725.73 ms/batch
Step 00001450 L_out=0.271 Acc=0.38 ---54282.37 ms/batch
Step 00001500 L_out=1.317 Acc=0.63 ---77555.25 ms/batch
Step 00001550 L_out=0.553 Acc=0.56 ---100807.07 ms/batch
Step 00001600 L_out=0.571 Acc=0.72 ---124397.26 ms/batch
Step 00001650 L_out=0.850 Acc=0.45 ---147774.13 ms/batch
Step 00001700 L_out=0.227 Acc=0.54 ---171165.14 ms/batch
Step 00001750 L_out=0.468 Acc=0.56 ---194553.87 ms/batch
Step 00001800 L_out=0.517 Acc=0.64 ---217795.86 ms/batch
Step 00001850 L_out=0.591 Acc=0.55 ---241176.75 ms/batch
Step 00001900 L_out=0.316 Acc=0.65 ---264411.27 ms/batch
Step 00001950 L_out=0.159 Acc=0.63 ---287665.40 ms/batch
Step 00002000 L_out=0.257 Acc=0.68 ---311141.46 ms/batch
Round 2 | epoch=2, train costTime=311.6297698020935, | total_activation_sum=12003432
Round 2 | ****EPOCH 3****
Step 00002050 L_out=0.494 Acc=0.60 ---23066.99 ms/batch
Step 00002100 L_out=0.931 Acc=0.58 ---46499.11 ms/batch
Step 00002150 L_out=0.181 Acc=0.54 ---69912.32 ms/batch
Step 00002200 L_out=1.156 Acc=0.60 ---93347.10 ms/batch
Step 00002250 L_out=0.512 Acc=0.60 ---116637.52 ms/batch
Step 00002300 L_out=0.798 Acc=0.76 ---140280.39 ms/batch
Step 00002350 L_out=0.180 Acc=0.59 ---163542.52 ms/batch
Step 00002400 L_out=0.589 Acc=0.79 ---186899.32 ms/batch
Step 00002450 L_out=0.642 Acc=0.62 ---210263.76 ms/batch
Step 00002500 L_out=0.227 Acc=0.68 ---233715.94 ms/batch
Step 00002550 L_out=0.734 Acc=0.75 ---257151.95 ms/batch
Step 00002600 L_out=0.301 Acc=0.70 ---280637.82 ms/batch
Step 00002650 L_out=0.238 Acc=0.74 ---304093.52 ms/batch
Round 2 | epoch=3, train costTime=312.5141270160675, | total_activation_sum=12267081
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.204 Acc=0.66 ---14834.98 ms/batch
Step 00002750 L_out=0.522 Acc=0.65 ---38313.16 ms/batch
Step 00002800 L_out=0.461 Acc=0.66 ---61734.25 ms/batch
Step 00002850 L_out=0.343 Acc=0.66 ---85120.49 ms/batch
Step 00002900 L_out=0.855 Acc=0.53 ---108609.78 ms/batch
Step 00002950 L_out=0.884 Acc=0.65 ---132063.49 ms/batch
Step 00003000 L_out=0.831 Acc=0.84 ---155660.75 ms/batch
Step 00003050 L_out=0.413 Acc=0.64 ---178998.87 ms/batch
Step 00003100 L_out=0.215 Acc=0.67 ---202655.36 ms/batch
Step 00003150 L_out=0.545 Acc=0.52 ---226089.32 ms/batch
Step 00003200 L_out=0.583 Acc=0.74 ---249445.19 ms/batch
Step 00003250 L_out=0.352 Acc=0.64 ---272827.96 ms/batch
Step 00003300 L_out=0.822 Acc=0.68 ---296203.90 ms/batch
Round 2 | epoch=4, train costTime=312.5256850719452, | total_activation_sum=12360100
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.194 Acc=0.64 --- 7053.38 ms/batch
Step 00003400 L_out=0.330 Acc=0.56 ---30575.39 ms/batch
Step 00003450 L_out=0.459 Acc=0.74 ---53981.18 ms/batch
Step 00003500 L_out=0.415 Acc=0.70 ---77495.22 ms/batch
Step 00003550 L_out=0.220 Acc=0.68 ---100761.10 ms/batch
Step 00003600 L_out=0.160 Acc=0.74 ---123952.29 ms/batch
Step 00003650 L_out=0.322 Acc=0.63 ---147199.21 ms/batch
Step 00003700 L_out=0.250 Acc=0.77 ---170665.41 ms/batch
Step 00003750 L_out=0.402 Acc=0.65 ---194035.70 ms/batch
Step 00003800 L_out=0.315 Acc=0.69 ---217425.99 ms/batch
Step 00003850 L_out=0.655 Acc=0.68 ---240766.85 ms/batch
Step 00003900 L_out=0.215 Acc=0.71 ---264159.24 ms/batch
Step 00003950 L_out=0.423 Acc=0.68 ---287667.64 ms/batch
Step 00004000 L_out=0.574 Acc=0.70 ---310903.34 ms/batch
Round 2 | epoch=5, train costTime=311.8649306297302, | total_activation_sum=12264031
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.487 Acc=0.82 ---22469.27 ms/batch
Step 00004100 L_out=0.191 Acc=0.78 ---45869.31 ms/batch
Step 00004150 L_out=0.216 Acc=0.72 ---69141.19 ms/batch
Step 00004200 L_out=0.523 Acc=0.70 ---92691.15 ms/batch
Step 00004250 L_out=0.379 Acc=0.77 ---115917.54 ms/batch
Step 00004300 L_out=0.272 Acc=0.85 ---139229.96 ms/batch
Step 00004350 L_out=0.399 Acc=0.85 ---162590.23 ms/batch
Step 00004400 L_out=0.560 Acc=0.70 ---186069.50 ms/batch
Step 00004450 L_out=0.176 Acc=0.78 ---209409.09 ms/batch
Step 00004500 L_out=0.240 Acc=0.62 ---232832.42 ms/batch
Step 00004550 L_out=0.541 Acc=0.62 ---256393.82 ms/batch
Step 00004600 L_out=0.278 Acc=0.68 ---279739.32 ms/batch
Step 00004650 L_out=0.141 Acc=0.73 ---303141.53 ms/batch
Round 2 | epoch=6, train costTime=311.95089840888977, | total_activation_sum=12281987
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.266 Acc=0.57 ---14473.28 ms/batch
Step 00004750 L_out=0.279 Acc=0.58 ---37665.41 ms/batch
Step 00004800 L_out=0.219 Acc=0.83 ---61209.13 ms/batch
Step 00004850 L_out=0.430 Acc=0.71 ---84844.50 ms/batch
Step 00004900 L_out=0.427 Acc=0.75 ---108051.77 ms/batch
Step 00004950 L_out=0.325 Acc=0.77 ---131488.21 ms/batch
Step 00005000 L_out=0.682 Acc=0.85 ---154844.25 ms/batch
Step 00005050 L_out=0.054 Acc=0.68 ---178251.86 ms/batch
Step 00005100 L_out=0.102 Acc=0.77 ---201703.59 ms/batch
Step 00005150 L_out=0.296 Acc=0.67 ---225034.79 ms/batch
Step 00005200 L_out=0.076 Acc=0.68 ---248606.65 ms/batch
Step 00005250 L_out=0.290 Acc=0.74 ---271955.52 ms/batch
Step 00005300 L_out=0.216 Acc=0.81 ---295418.06 ms/batch
Round 2 | epoch=7, train costTime=312.14439272880554, | total_activation_sum=12297123
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.534 Acc=0.78 --- 6586.92 ms/batch
Step 00005400 L_out=0.219 Acc=0.63 ---29913.17 ms/batch
Step 00005450 L_out=0.272 Acc=0.76 ---53318.71 ms/batch
Step 00005500 L_out=0.198 Acc=0.57 ---76754.39 ms/batch
Step 00005550 L_out=0.365 Acc=0.66 ---99972.88 ms/batch
Step 00005600 L_out=0.504 Acc=0.77 ---123488.68 ms/batch
Step 00005650 L_out=0.201 Acc=0.70 ---147134.80 ms/batch
Step 00005700 L_out=0.586 Acc=0.83 ---170744.78 ms/batch
Step 00005750 L_out=0.192 Acc=0.81 ---194231.53 ms/batch
Step 00005800 L_out=0.387 Acc=0.74 ---217676.81 ms/batch
Step 00005850 L_out=0.221 Acc=0.72 ---241013.52 ms/batch
Step 00005900 L_out=0.434 Acc=0.65 ---264408.64 ms/batch
Step 00005950 L_out=0.754 Acc=0.78 ---287769.46 ms/batch
Step 00006000 L_out=0.413 Acc=0.90 ---311123.85 ms/batch
Round 2 | epoch=8, train costTime=312.4939079284668, | total_activation_sum=12365001
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.171 Acc=0.79 ---21807.93 ms/batch
Step 00006100 L_out=0.493 Acc=0.77 ---45371.12 ms/batch
Step 00006150 L_out=0.320 Acc=0.74 ---68803.73 ms/batch
Step 00006200 L_out=0.774 Acc=0.77 ---92056.53 ms/batch
Step 00006250 L_out=0.173 Acc=0.76 ---115585.09 ms/batch
Step 00006300 L_out=0.366 Acc=0.76 ---138947.33 ms/batch
Step 00006350 L_out=0.205 Acc=0.67 ---162465.05 ms/batch
Step 00006400 L_out=0.253 Acc=0.83 ---185775.18 ms/batch
Step 00006450 L_out=0.108 Acc=0.77 ---209247.70 ms/batch
Step 00006500 L_out=0.376 Acc=0.79 ---232748.47 ms/batch
Step 00006550 L_out=0.655 Acc=0.82 ---256220.69 ms/batch
Step 00006600 L_out=0.208 Acc=0.81 ---279631.29 ms/batch
Step 00006650 L_out=0.365 Acc=0.82 ---302952.02 ms/batch
Round 2 | epoch=9, train costTime=312.3410542011261, | total_activation_sum=12194474
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.113 Acc=0.79 ---14035.08 ms/batch
Step 00006750 L_out=0.173 Acc=0.77 ---37442.70 ms/batch
Step 00006800 L_out=0.237 Acc=0.68 ---60881.18 ms/batch
Step 00006850 L_out=0.340 Acc=0.82 ---84215.72 ms/batch
Step 00006900 L_out=0.333 Acc=0.78 ---107503.24 ms/batch
Step 00006950 L_out=0.350 Acc=0.75 ---131035.29 ms/batch
Step 00007000 L_out=0.998 Acc=0.66 ---154326.98 ms/batch
Step 00007050 L_out=0.344 Acc=0.56 ---177769.33 ms/batch
Step 00007100 L_out=0.423 Acc=0.78 ---201179.63 ms/batch
Step 00007150 L_out=0.145 Acc=0.71 ---224523.18 ms/batch
Step 00007200 L_out=0.190 Acc=0.85 ---248005.84 ms/batch
Step 00007250 L_out=0.358 Acc=0.74 ---271363.13 ms/batch
Step 00007300 L_out=0.248 Acc=0.65 ---294561.56 ms/batch
Round 2 | epoch=10, train costTime=311.9281792640686, | total_activation_sum=12262145
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.165 Acc=0.87 --- 6082.61 ms/batch
Step 00007400 L_out=0.141 Acc=0.88 ---29535.85 ms/batch
Step 00007450 L_out=0.297 Acc=0.79 ---53001.37 ms/batch
Step 00007500 L_out=0.493 Acc=0.86 ---76453.91 ms/batch
Step 00007550 L_out=0.179 Acc=0.86 ---99878.20 ms/batch
Step 00007600 L_out=0.318 Acc=0.74 ---123277.79 ms/batch
Step 00007650 L_out=0.288 Acc=0.70 ---146682.83 ms/batch
Step 00007700 L_out=0.029 Acc=0.74 ---170095.18 ms/batch
Step 00007750 L_out=0.512 Acc=0.83 ---193488.64 ms/batch
Step 00007800 L_out=0.248 Acc=0.94 ---216974.26 ms/batch
Step 00007850 L_out=0.191 Acc=0.67 ---240419.02 ms/batch
Step 00007900 L_out=0.254 Acc=0.80 ---263812.40 ms/batch
Step 00007950 L_out=0.091 Acc=0.88 ---287331.48 ms/batch
Step 00008000 L_out=0.230 Acc=0.67 ---310613.62 ms/batch
Round 2 | epoch=11, train costTime=312.45404982566833, | total_activation_sum=12197352
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.335 Acc=0.75 ---21513.00 ms/batch
Step 00008100 L_out=0.363 Acc=0.74 ---44849.75 ms/batch
Step 00008150 L_out=0.340 Acc=0.77 ---68081.06 ms/batch
Step 00008200 L_out=0.241 Acc=0.83 ---91522.93 ms/batch
Step 00008250 L_out=0.248 Acc=0.78 ---114985.35 ms/batch
Step 00008300 L_out=0.078 Acc=0.67 ---138418.37 ms/batch
Step 00008350 L_out=0.340 Acc=0.75 ---161846.49 ms/batch
Step 00008400 L_out=0.298 Acc=0.75 ---185310.69 ms/batch
Step 00008450 L_out=1.717 Acc=0.84 ---208648.85 ms/batch
Step 00008500 L_out=0.138 Acc=0.89 ---232010.41 ms/batch
Step 00008550 L_out=0.163 Acc=0.83 ---255508.76 ms/batch
Step 00008600 L_out=0.187 Acc=0.81 ---279075.83 ms/batch
Step 00008650 L_out=0.267 Acc=0.81 ---302528.18 ms/batch
Round 2 | epoch=12, train costTime=312.4562442302704, | total_activation_sum=12378921
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.452 Acc=0.87 ---13528.64 ms/batch
Step 00008750 L_out=0.125 Acc=0.80 ---36882.25 ms/batch
Step 00008800 L_out=0.052 Acc=0.86 ---60347.60 ms/batch
Step 00008850 L_out=0.312 Acc=0.87 ---83866.50 ms/batch
Step 00008900 L_out=0.095 Acc=0.84 ---107311.00 ms/batch
Step 00008950 L_out=0.431 Acc=0.81 ---130517.80 ms/batch
Step 00009000 L_out=0.107 Acc=0.78 ---153792.10 ms/batch
Step 00009050 L_out=0.602 Acc=0.55 ---177254.11 ms/batch
Step 00009100 L_out=0.407 Acc=0.79 ---200669.95 ms/batch
Step 00009150 L_out=0.272 Acc=0.86 ---223997.30 ms/batch
Step 00009200 L_out=0.211 Acc=0.84 ---247431.46 ms/batch
Step 00009250 L_out=0.194 Acc=0.88 ---270759.50 ms/batch
Step 00009300 L_out=0.209 Acc=0.85 ---294443.37 ms/batch
Round 2 | epoch=13, train costTime=312.1380023956299, | total_activation_sum=12399230
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.329 Acc=0.83 --- 5653.67 ms/batch
Step 00009400 L_out=0.202 Acc=0.69 ---29080.61 ms/batch
Step 00009450 L_out=0.477 Acc=0.73 ---52377.23 ms/batch
Step 00009500 L_out=0.382 Acc=0.71 ---75765.95 ms/batch
Step 00009550 L_out=0.279 Acc=0.82 ---99027.17 ms/batch
Step 00009600 L_out=0.036 Acc=0.84 ---122303.55 ms/batch
Step 00009650 L_out=0.146 Acc=0.84 ---145663.59 ms/batch
Step 00009700 L_out=0.265 Acc=0.69 ---169197.08 ms/batch
Step 00009750 L_out=0.233 Acc=0.83 ---192557.82 ms/batch
Step 00009800 L_out=0.311 Acc=0.83 ---215803.71 ms/batch
Step 00009850 L_out=0.147 Acc=0.89 ---239183.22 ms/batch
Step 00009900 L_out=0.401 Acc=0.78 ---262533.80 ms/batch
Step 00009950 L_out=0.240 Acc=0.89 ---285980.97 ms/batch
Step 00010000 L_out=0.068 Acc=0.73 ---309292.21 ms/batch
Round 2 | epoch=14, train costTime=311.6579897403717, | total_activation_sum=12181403
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.214 Acc=0.81 ---20985.03 ms/batch
Step 00010100 L_out=0.075 Acc=0.92 ---44266.05 ms/batch
Step 00010150 L_out=0.118 Acc=0.85 ---67519.48 ms/batch
Step 00010200 L_out=0.200 Acc=0.88 ---90914.45 ms/batch
Step 00010250 L_out=0.174 Acc=0.82 ---114347.71 ms/batch
Step 00010300 L_out=0.085 Acc=0.80 ---137843.65 ms/batch
Step 00010350 L_out=0.218 Acc=0.81 ---161276.60 ms/batch
Step 00010400 L_out=0.211 Acc=0.84 ---184461.89 ms/batch
Step 00010450 L_out=0.162 Acc=0.90 ---207788.38 ms/batch
Step 00010500 L_out=0.081 Acc=0.79 ---231344.99 ms/batch
Step 00010550 L_out=0.516 Acc=0.88 ---254634.59 ms/batch
Step 00010600 L_out=0.222 Acc=0.76 ---278130.39 ms/batch
Step 00010650 L_out=0.262 Acc=0.74 ---301477.98 ms/batch
Round 2 | epoch=15, train costTime=311.7331943511963, | total_activation_sum=12468062
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.246 Acc=0.83 ---13059.21 ms/batch
Step 00010750 L_out=0.141 Acc=0.87 ---36432.77 ms/batch
Step 00010800 L_out=0.209 Acc=0.91 ---59716.09 ms/batch
Step 00010850 L_out=0.080 Acc=0.81 ---82994.99 ms/batch
Step 00010900 L_out=0.055 Acc=0.87 ---106275.59 ms/batch
Step 00010950 L_out=0.432 Acc=0.86 ---129569.48 ms/batch
Step 00011000 L_out=0.259 Acc=0.89 ---152861.65 ms/batch
Step 00011050 L_out=0.256 Acc=0.80 ---176375.92 ms/batch
Step 00011100 L_out=0.065 Acc=0.75 ---199811.33 ms/batch
Step 00011150 L_out=0.580 Acc=0.87 ---223166.25 ms/batch
Step 00011200 L_out=0.118 Acc=0.81 ---246355.03 ms/batch
Step 00011250 L_out=0.118 Acc=0.80 ---269658.67 ms/batch
Step 00011300 L_out=0.184 Acc=0.90 ---293078.27 ms/batch
Round 2 | epoch=16, train costTime=311.423499584198, | total_activation_sum=12361375
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.239 Acc=0.91 --- 5093.00 ms/batch
Step 00011400 L_out=0.295 Acc=0.75 ---28298.57 ms/batch
Step 00011450 L_out=0.511 Acc=0.84 ---51820.44 ms/batch
Step 00011500 L_out=0.119 Acc=0.81 ---75100.39 ms/batch
Step 00011550 L_out=0.396 Acc=0.80 ---98543.06 ms/batch
Step 00011600 L_out=0.266 Acc=0.87 ---121796.51 ms/batch
Step 00011650 L_out=0.154 Acc=0.84 ---145330.66 ms/batch
Step 00011700 L_out=0.086 Acc=0.89 ---168862.39 ms/batch
Step 00011750 L_out=0.143 Acc=0.88 ---192045.80 ms/batch
Step 00011800 L_out=0.064 Acc=0.89 ---215343.30 ms/batch
Step 00011850 L_out=0.219 Acc=0.86 ---238666.88 ms/batch
Step 00011900 L_out=0.555 Acc=0.77 ---261949.84 ms/batch
Step 00011950 L_out=0.284 Acc=0.79 ---285546.19 ms/batch
Step 00012000 L_out=1.606 Acc=0.77 ---308967.33 ms/batch
Round 2 | epoch=17, train costTime=311.77336072921753, | total_activation_sum=12123654
Round 2 | epoch=18, current m_iou=0.49085328321220223
Round 2 | Best m_IoU is: 0.491, OA is: 0.818 | val costTime=117.68970608711243
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.210 Acc=0.86 ---20838.28 ms/batch
Step 00012100 L_out=0.216 Acc=0.71 ---44240.23 ms/batch
Step 00012150 L_out=0.187 Acc=0.88 ---67588.33 ms/batch
Step 00012200 L_out=0.111 Acc=0.78 ---90813.70 ms/batch
Step 00012250 L_out=0.095 Acc=0.89 ---114093.21 ms/batch
Step 00012300 L_out=0.160 Acc=0.92 ---137466.75 ms/batch
Step 00012350 L_out=0.097 Acc=0.80 ---160979.83 ms/batch
Step 00012400 L_out=0.150 Acc=0.78 ---184185.13 ms/batch
Step 00012450 L_out=0.087 Acc=0.92 ---207514.11 ms/batch
Step 00012500 L_out=0.376 Acc=0.85 ---230790.38 ms/batch
Step 00012550 L_out=0.174 Acc=0.88 ---254293.19 ms/batch
Step 00012600 L_out=0.143 Acc=0.82 ---277608.62 ms/batch
Step 00012650 L_out=0.258 Acc=0.76 ---300764.69 ms/batch
Round 2 | epoch=18, train costTime=311.50538635253906, | total_activation_sum=12281058
Round 2 | epoch=19, current m_iou=0.5622666219128452
Round 2 | Best m_IoU is: 0.562, OA is: 0.866 | val costTime=117.71842765808105
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.196 Acc=0.84 ---12724.16 ms/batch
Step 00012750 L_out=0.936 Acc=0.85 ---36019.57 ms/batch
Step 00012800 L_out=0.230 Acc=0.77 ---59175.87 ms/batch
Step 00012850 L_out=0.126 Acc=0.90 ---82601.48 ms/batch
Step 00012900 L_out=0.636 Acc=0.80 ---105701.48 ms/batch
Step 00012950 L_out=0.141 Acc=0.84 ---129131.16 ms/batch
Step 00013000 L_out=0.097 Acc=0.85 ---152385.88 ms/batch
Step 00013050 L_out=0.208 Acc=0.90 ---175641.14 ms/batch
Step 00013100 L_out=0.129 Acc=0.94 ---198942.40 ms/batch
Step 00013150 L_out=0.079 Acc=0.91 ---222316.65 ms/batch
Step 00013200 L_out=0.381 Acc=0.69 ---245397.46 ms/batch
Step 00013250 L_out=0.402 Acc=0.87 ---268660.93 ms/batch
Step 00013300 L_out=0.097 Acc=0.86 ---292107.97 ms/batch
Round 2 | epoch=19, train costTime=310.7452630996704, | total_activation_sum=12423117
Round 2 | epoch=20, current m_iou=0.6019429391968973
Round 2 | Best m_IoU is: 0.602, OA is: 0.892 | val costTime=117.07994818687439
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.046 Acc=0.89 --- 4731.16 ms/batch
Step 00013400 L_out=0.316 Acc=0.78 ---27994.22 ms/batch
Step 00013450 L_out=0.098 Acc=0.87 ---51441.12 ms/batch
Step 00013500 L_out=0.150 Acc=0.83 ---74654.01 ms/batch
Step 00013550 L_out=0.134 Acc=0.73 ---97893.14 ms/batch
Step 00013600 L_out=0.080 Acc=0.89 ---121441.69 ms/batch
Step 00013650 L_out=0.078 Acc=0.85 ---144577.73 ms/batch
Step 00013700 L_out=0.124 Acc=0.80 ---167702.15 ms/batch
Step 00013750 L_out=1.336 Acc=0.59 ---191136.13 ms/batch
Step 00013800 L_out=0.090 Acc=0.84 ---214527.87 ms/batch
Step 00013850 L_out=0.038 Acc=0.92 ---237900.82 ms/batch
Step 00013900 L_out=0.333 Acc=0.88 ---261458.40 ms/batch
Step 00013950 L_out=0.114 Acc=0.93 ---284954.19 ms/batch
Step 00014000 L_out=0.043 Acc=0.81 ---308499.57 ms/batch
Round 2 | epoch=20, train costTime=311.7523844242096, | total_activation_sum=12358238
Round 2 | epoch=21, current m_iou=0.6023570875905738
Round 2 | Best m_IoU is: 0.602, OA is: 0.893 | val costTime=117.63417315483093
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.055 Acc=0.94 ---20195.34 ms/batch
Step 00014100 L_out=0.362 Acc=0.85 ---43627.77 ms/batch
Step 00014150 L_out=0.126 Acc=0.82 ---67014.64 ms/batch
Step 00014200 L_out=0.116 Acc=0.90 ---90464.08 ms/batch
Step 00014250 L_out=0.275 Acc=0.80 ---113694.88 ms/batch
Step 00014300 L_out=0.085 Acc=0.85 ---136875.55 ms/batch
Step 00014350 L_out=0.058 Acc=0.93 ---160225.07 ms/batch
Step 00014400 L_out=0.194 Acc=0.84 ---183671.20 ms/batch
Step 00014450 L_out=0.802 Acc=0.80 ---207161.74 ms/batch
Step 00014500 L_out=0.186 Acc=0.74 ---230175.25 ms/batch
Step 00014550 L_out=0.092 Acc=0.79 ---253463.15 ms/batch
Step 00014600 L_out=0.172 Acc=0.86 ---276769.98 ms/batch
Step 00014650 L_out=0.068 Acc=0.83 ---300145.40 ms/batch
Round 2 | epoch=21, train costTime=311.3792541027069, | total_activation_sum=12158537
Round 2 | epoch=22, current m_iou=0.5918429851245843
Round 2 | Best m_IoU is: 0.602, OA is: 0.893 | val costTime=117.1810851097107
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.085 Acc=0.81 ---12151.83 ms/batch
Step 00014750 L_out=0.181 Acc=0.88 ---35565.18 ms/batch
Step 00014800 L_out=0.244 Acc=0.80 ---58791.41 ms/batch
Step 00014850 L_out=0.048 Acc=0.92 ---82277.94 ms/batch
Step 00014900 L_out=0.093 Acc=0.88 ---105679.55 ms/batch
Step 00014950 L_out=0.281 Acc=0.83 ---129085.67 ms/batch
Step 00015000 L_out=0.421 Acc=0.79 ---152267.19 ms/batch
Step 00015050 L_out=0.060 Acc=0.86 ---175686.55 ms/batch
Step 00015100 L_out=0.212 Acc=0.91 ---199068.61 ms/batch
Step 00015150 L_out=0.088 Acc=0.91 ---222437.41 ms/batch
Step 00015200 L_out=0.153 Acc=0.91 ---245760.73 ms/batch
Step 00015250 L_out=0.144 Acc=0.90 ---269312.07 ms/batch
Step 00015300 L_out=0.072 Acc=0.77 ---292513.52 ms/batch
Round 2 | epoch=22, train costTime=311.78161549568176, | total_activation_sum=12317412
Round 2 | epoch=23, current m_iou=0.5957980556346104
Round 2 | Best m_IoU is: 0.602, OA is: 0.893 | val costTime=116.84179282188416
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.120 Acc=0.93 --- 4237.04 ms/batch
Step 00015400 L_out=0.155 Acc=0.91 ---27595.55 ms/batch
Step 00015450 L_out=0.083 Acc=0.78 ---50843.61 ms/batch
Step 00015500 L_out=0.088 Acc=0.90 ---74255.46 ms/batch
Step 00015550 L_out=0.212 Acc=0.88 ---97677.87 ms/batch
Step 00015600 L_out=0.112 Acc=0.88 ---121077.11 ms/batch
Step 00015650 L_out=0.074 Acc=0.88 ---144469.39 ms/batch
Step 00015700 L_out=0.080 Acc=0.92 ---167982.38 ms/batch
Step 00015750 L_out=0.045 Acc=0.89 ---191392.40 ms/batch
Step 00015800 L_out=0.196 Acc=0.83 ---214805.94 ms/batch
Step 00015850 L_out=0.068 Acc=0.90 ---238581.20 ms/batch
Step 00015900 L_out=0.070 Acc=0.90 ---261874.90 ms/batch
Step 00015950 L_out=0.064 Acc=0.82 ---285096.63 ms/batch
Step 00016000 L_out=0.221 Acc=0.85 ---308583.33 ms/batch
Round 2 | epoch=23, train costTime=312.21826815605164, | total_activation_sum=12317838
Round 2 | epoch=24, current m_iou=0.6060268315577492
Round 2 | Best m_IoU is: 0.606, OA is: 0.891 | val costTime=117.58016276359558
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.093 Acc=0.88 ---19692.41 ms/batch
Step 00016100 L_out=0.039 Acc=0.92 ---43209.46 ms/batch
Step 00016150 L_out=0.230 Acc=0.84 ---66483.39 ms/batch
Step 00016200 L_out=0.150 Acc=0.82 ---89497.05 ms/batch
Step 00016250 L_out=0.048 Acc=0.93 ---112843.45 ms/batch
Step 00016300 L_out=0.057 Acc=0.84 ---136128.63 ms/batch
Step 00016350 L_out=0.077 Acc=0.89 ---159582.53 ms/batch
Step 00016400 L_out=0.123 Acc=0.91 ---182882.46 ms/batch
Step 00016450 L_out=0.423 Acc=0.87 ---206231.92 ms/batch
Step 00016500 L_out=0.083 Acc=0.94 ---229613.16 ms/batch
Step 00016550 L_out=0.151 Acc=0.89 ---252839.79 ms/batch
Step 00016600 L_out=0.134 Acc=0.80 ---276349.71 ms/batch
Step 00016650 L_out=0.140 Acc=0.91 ---299686.30 ms/batch
Round 2 | epoch=24, train costTime=311.3966019153595, | total_activation_sum=12330647
Round 2 | epoch=25, current m_iou=0.6072469170656178
Round 2 | Best m_IoU is: 0.607, OA is: 0.894 | val costTime=118.0088095664978
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.080 Acc=0.90 ---11656.92 ms/batch
Step 00016750 L_out=0.247 Acc=0.79 ---34975.81 ms/batch
Step 00016800 L_out=0.163 Acc=0.85 ---58407.33 ms/batch
Step 00016850 L_out=0.302 Acc=0.87 ---81626.61 ms/batch
Step 00016900 L_out=0.282 Acc=0.84 ---104883.77 ms/batch
Step 00016950 L_out=0.088 Acc=0.92 ---128344.96 ms/batch
Step 00017000 L_out=0.465 Acc=0.82 ---151888.46 ms/batch
Step 00017050 L_out=0.081 Acc=0.78 ---175162.03 ms/batch
Step 00017100 L_out=0.171 Acc=0.94 ---198491.18 ms/batch
Step 00017150 L_out=0.299 Acc=0.87 ---221965.47 ms/batch
Step 00017200 L_out=0.567 Acc=0.78 ---245229.22 ms/batch
Step 00017250 L_out=0.029 Acc=0.90 ---268737.32 ms/batch
Step 00017300 L_out=0.305 Acc=0.86 ---292049.39 ms/batch
Round 2 | epoch=25, train costTime=311.58845567703247, | total_activation_sum=12359473
Round 2 | epoch=26, current m_iou=0.616318149524792
Round 2 | Best m_IoU is: 0.616, OA is: 0.896 | val costTime=117.41901135444641
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.115 Acc=0.93 --- 3751.31 ms/batch
Step 00017400 L_out=0.127 Acc=0.89 ---27175.27 ms/batch
Step 00017450 L_out=0.079 Acc=0.91 ---50558.06 ms/batch
Step 00017500 L_out=0.123 Acc=0.90 ---73917.98 ms/batch
Step 00017550 L_out=0.072 Acc=0.95 ---97246.89 ms/batch
Step 00017600 L_out=0.157 Acc=0.90 ---120573.97 ms/batch
Step 00017650 L_out=0.137 Acc=0.91 ---143891.34 ms/batch
Step 00017700 L_out=0.425 Acc=0.90 ---167289.32 ms/batch
Step 00017750 L_out=0.109 Acc=0.85 ---190771.81 ms/batch
Step 00017800 L_out=0.071 Acc=0.86 ---214192.45 ms/batch
Step 00017850 L_out=0.123 Acc=0.92 ---237539.08 ms/batch
Step 00017900 L_out=0.426 Acc=0.87 ---260926.40 ms/batch
Step 00017950 L_out=0.049 Acc=0.90 ---284415.55 ms/batch
Step 00018000 L_out=0.124 Acc=0.93 ---308266.38 ms/batch
Round 2 | epoch=26, train costTime=312.4935805797577, | total_activation_sum=12191244
Round 2 | epoch=27, current m_iou=0.6228170203646852
Round 2 | Best m_IoU is: 0.623, OA is: 0.899 | val costTime=117.57695388793945
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.120 Acc=0.86 ---19228.63 ms/batch
Step 00018100 L_out=0.052 Acc=0.91 ---42466.13 ms/batch
Step 00018150 L_out=0.138 Acc=0.82 ---65815.88 ms/batch
Step 00018200 L_out=0.150 Acc=0.86 ---89133.65 ms/batch
Step 00018250 L_out=0.131 Acc=0.85 ---112771.13 ms/batch
Step 00018300 L_out=0.260 Acc=0.78 ---136289.64 ms/batch
Step 00018350 L_out=0.369 Acc=0.80 ---159509.85 ms/batch
Step 00018400 L_out=0.201 Acc=0.86 ---182736.85 ms/batch
Step 00018450 L_out=0.120 Acc=0.84 ---206173.89 ms/batch
Step 00018500 L_out=0.068 Acc=0.92 ---229620.71 ms/batch
Step 00018550 L_out=0.229 Acc=0.85 ---252910.77 ms/batch
Step 00018600 L_out=0.098 Acc=0.88 ---276316.07 ms/batch
Step 00018650 L_out=0.153 Acc=0.85 ---299611.41 ms/batch
Round 2 | epoch=27, train costTime=311.69008469581604, | total_activation_sum=12305166
Round 2 | epoch=28, current m_iou=0.6224606694372701
Round 2 | Best m_IoU is: 0.623, OA is: 0.899 | val costTime=117.34191727638245
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.151 Acc=0.91 ---11251.16 ms/batch
Step 00018750 L_out=0.071 Acc=0.88 ---34551.54 ms/batch
Step 00018800 L_out=0.038 Acc=0.85 ---58191.92 ms/batch
Step 00018850 L_out=0.207 Acc=0.90 ---81305.74 ms/batch
Step 00018900 L_out=0.356 Acc=0.89 ---104728.88 ms/batch
Step 00018950 L_out=0.090 Acc=0.92 ---128217.36 ms/batch
Step 00019000 L_out=0.044 Acc=0.86 ---151854.25 ms/batch
Step 00019050 L_out=0.046 Acc=0.87 ---175081.95 ms/batch
Step 00019100 L_out=0.151 Acc=0.89 ---198460.72 ms/batch
Step 00019150 L_out=0.198 Acc=0.83 ---221855.01 ms/batch
Step 00019200 L_out=0.150 Acc=0.88 ---245445.89 ms/batch
Step 00019250 L_out=0.082 Acc=0.90 ---268847.09 ms/batch
Step 00019300 L_out=0.093 Acc=0.95 ---292138.83 ms/batch
Round 2 | epoch=28, train costTime=312.0076868534088, | total_activation_sum=12179697
Round 2 | epoch=29, current m_iou=0.6223564766665451
Round 2 | Best m_IoU is: 0.623, OA is: 0.899 | val costTime=117.88778114318848
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.134 Acc=0.88 --- 3292.67 ms/batch
Step 00019400 L_out=0.075 Acc=0.94 ---26622.89 ms/batch
Step 00019450 L_out=0.142 Acc=0.78 ---50003.65 ms/batch
Step 00019500 L_out=0.164 Acc=0.82 ---73324.07 ms/batch
Step 00019550 L_out=0.091 Acc=0.80 ---96771.88 ms/batch
Step 00019600 L_out=0.087 Acc=0.74 ---120321.90 ms/batch
Step 00019650 L_out=0.422 Acc=0.85 ---143607.31 ms/batch
Step 00019700 L_out=0.078 Acc=0.81 ---167047.88 ms/batch
Step 00019750 L_out=0.150 Acc=0.83 ---190537.28 ms/batch
Step 00019800 L_out=0.136 Acc=0.92 ---213982.02 ms/batch
Step 00019850 L_out=0.055 Acc=0.89 ---237468.79 ms/batch
Step 00019900 L_out=0.151 Acc=0.88 ---260920.25 ms/batch
Step 00019950 L_out=0.084 Acc=0.89 ---284292.33 ms/batch
Step 00020000 L_out=0.065 Acc=0.90 ---307598.83 ms/batch
Round 2 | epoch=29, train costTime=312.33052110671997, | total_activation_sum=12584987
Round 2 | epoch=30, current m_iou=0.6235113657669942
Round 2 | Best m_IoU is: 0.624, OA is: 0.903 | val costTime=117.48770380020142
Round 2 | ****EPOCH 30****
Round 3 | ****EPOCH 0****
Step 00020050 L_out=0.607 Acc=0.64 ---18797.69 ms/batch
Step 00020100 L_out=1.819 Acc=0.63 ---42164.74 ms/batch
Step 00020150 L_out=0.541 Acc=0.68 ---65702.88 ms/batch
Step 00020200 L_out=0.609 Acc=0.79 ---89301.33 ms/batch
Step 00020250 L_out=0.763 Acc=0.71 ---112746.65 ms/batch
Step 00020300 L_out=0.490 Acc=0.89 ---136269.71 ms/batch
Step 00020350 L_out=0.501 Acc=0.83 ---159902.28 ms/batch
Step 00020400 L_out=0.721 Acc=0.55 ---183245.40 ms/batch
Step 00020450 L_out=0.347 Acc=0.82 ---206719.31 ms/batch
Step 00020500 L_out=0.161 Acc=0.90 ---230119.87 ms/batch
Step 00020550 L_out=0.659 Acc=0.78 ---253331.24 ms/batch
Step 00020600 L_out=0.916 Acc=0.87 ---276778.61 ms/batch
Step 00020650 L_out=0.270 Acc=0.85 ---300110.53 ms/batch
Round 3 | epoch=0, train costTime=312.7366740703583, | total_activation_sum=22184648
Round 3 | ****EPOCH 1****
Step 00020700 L_out=0.706 Acc=0.78 ---10658.06 ms/batch
Step 00020750 L_out=1.583 Acc=0.43 ---34049.60 ms/batch
Step 00020800 L_out=0.511 Acc=0.65 ---57477.48 ms/batch
Step 00020850 L_out=0.907 Acc=0.77 ---80825.69 ms/batch
Step 00020900 L_out=0.258 Acc=0.83 ---104018.87 ms/batch
Step 00020950 L_out=0.581 Acc=0.75 ---127372.44 ms/batch
Step 00021000 L_out=1.592 Acc=0.80 ---150758.29 ms/batch
Step 00021050 L_out=0.870 Acc=0.71 ---174081.52 ms/batch
Step 00021100 L_out=0.589 Acc=0.80 ---197255.77 ms/batch
Step 00021150 L_out=0.414 Acc=0.81 ---220727.31 ms/batch
Step 00021200 L_out=0.526 Acc=0.79 ---244227.63 ms/batch
Step 00021250 L_out=1.008 Acc=0.83 ---267615.68 ms/batch
Step 00021300 L_out=0.687 Acc=0.82 ---291016.29 ms/batch
Round 3 | epoch=1, train costTime=311.6857633590698, | total_activation_sum=20422034
Round 3 | ****EPOCH 2****
Step 00021350 L_out=0.526 Acc=0.86 --- 2782.95 ms/batch
Step 00021400 L_out=0.194 Acc=0.88 ---26214.23 ms/batch
Step 00021450 L_out=0.379 Acc=0.90 ---49790.02 ms/batch
Step 00021500 L_out=0.646 Acc=0.83 ---73259.46 ms/batch
Step 00021550 L_out=0.462 Acc=0.71 ---96740.53 ms/batch
Step 00021600 L_out=0.138 Acc=0.84 ---120251.05 ms/batch
Step 00021650 L_out=0.218 Acc=0.93 ---143724.20 ms/batch
Step 00021700 L_out=1.074 Acc=0.76 ---167050.65 ms/batch
Step 00021750 L_out=0.227 Acc=0.89 ---190727.79 ms/batch
Step 00021800 L_out=0.852 Acc=0.84 ---214308.38 ms/batch
Step 00021850 L_out=2.311 Acc=0.49 ---237646.69 ms/batch
Step 00021900 L_out=0.725 Acc=0.78 ---261157.07 ms/batch
Step 00021950 L_out=0.876 Acc=0.74 ---284627.70 ms/batch
Step 00022000 L_out=0.307 Acc=0.89 ---308172.28 ms/batch
Round 3 | epoch=2, train costTime=313.35347986221313, | total_activation_sum=20536893
Round 3 | ****EPOCH 3****
Step 00022050 L_out=0.174 Acc=0.86 ---18353.64 ms/batch
Step 00022100 L_out=0.426 Acc=0.79 ---41918.56 ms/batch
Step 00022150 L_out=0.908 Acc=0.78 ---65338.14 ms/batch
Step 00022200 L_out=0.419 Acc=0.82 ---88515.73 ms/batch
Step 00022250 L_out=0.685 Acc=0.81 ---112023.54 ms/batch
Step 00022300 L_out=0.573 Acc=0.62 ---135273.38 ms/batch
Step 00022350 L_out=0.185 Acc=0.83 ---158683.63 ms/batch
Step 00022400 L_out=0.956 Acc=0.81 ---182163.31 ms/batch
Step 00022450 L_out=0.810 Acc=0.87 ---205501.10 ms/batch
Step 00022500 L_out=0.526 Acc=0.84 ---228983.65 ms/batch
Step 00022550 L_out=0.236 Acc=0.89 ---252231.17 ms/batch
Step 00022600 L_out=1.405 Acc=0.64 ---275588.37 ms/batch
Step 00022650 L_out=0.362 Acc=0.93 ---298909.50 ms/batch
Round 3 | epoch=3, train costTime=312.10765528678894, | total_activation_sum=20624156
Round 3 | ****EPOCH 4****
Step 00022700 L_out=0.346 Acc=0.76 ---10228.38 ms/batch
Step 00022750 L_out=0.408 Acc=0.85 ---33586.64 ms/batch
Step 00022800 L_out=0.337 Acc=0.84 ---56950.29 ms/batch
Step 00022850 L_out=1.008 Acc=0.78 ---80347.00 ms/batch
Step 00022900 L_out=0.484 Acc=0.84 ---103705.56 ms/batch
Step 00022950 L_out=0.358 Acc=0.90 ---127098.01 ms/batch
Step 00023000 L_out=0.149 Acc=0.86 ---150311.37 ms/batch
Step 00023050 L_out=0.175 Acc=0.87 ---173797.29 ms/batch
Step 00023100 L_out=0.488 Acc=0.83 ---197122.93 ms/batch
Step 00023150 L_out=2.824 Acc=0.74 ---220461.28 ms/batch
Step 00023200 L_out=0.656 Acc=0.83 ---243983.75 ms/batch
Step 00023250 L_out=0.867 Acc=0.82 ---267401.85 ms/batch
Step 00023300 L_out=0.317 Acc=0.91 ---290880.79 ms/batch
Round 3 | epoch=4, train costTime=311.82998061180115, | total_activation_sum=20501877
Round 3 | ****EPOCH 5****
Step 00023350 L_out=0.242 Acc=0.96 --- 2363.52 ms/batch
Step 00023400 L_out=0.667 Acc=0.87 ---25696.57 ms/batch
Step 00023450 L_out=0.382 Acc=0.83 ---49199.41 ms/batch
Step 00023500 L_out=1.918 Acc=0.84 ---72712.80 ms/batch
Step 00023550 L_out=1.115 Acc=0.74 ---96174.85 ms/batch
Step 00023600 L_out=0.708 Acc=0.86 ---119661.18 ms/batch
Step 00023650 L_out=1.233 Acc=0.77 ---142966.98 ms/batch
Step 00023700 L_out=1.154 Acc=0.78 ---166395.09 ms/batch
Step 00023750 L_out=0.487 Acc=0.86 ---189785.38 ms/batch
Step 00023800 L_out=0.097 Acc=0.94 ---212964.45 ms/batch
Step 00023850 L_out=0.536 Acc=0.73 ---236522.57 ms/batch
Step 00023900 L_out=0.504 Acc=0.80 ---259755.03 ms/batch
Step 00023950 L_out=0.314 Acc=0.83 ---283237.20 ms/batch
Step 00024000 L_out=0.420 Acc=0.83 ---306607.85 ms/batch
Round 3 | epoch=5, train costTime=312.2060980796814, | total_activation_sum=20451022
Round 3 | ****EPOCH 6****
Step 00024050 L_out=0.571 Acc=0.71 ---17825.82 ms/batch
Step 00024100 L_out=0.893 Acc=0.86 ---41152.06 ms/batch
Step 00024150 L_out=0.359 Acc=0.78 ---64414.10 ms/batch
Step 00024200 L_out=0.181 Acc=0.94 ---87736.53 ms/batch
Step 00024250 L_out=0.933 Acc=0.89 ---111363.90 ms/batch
Step 00024300 L_out=0.452 Acc=0.76 ---134735.75 ms/batch
Step 00024350 L_out=0.299 Acc=0.91 ---158239.71 ms/batch
Step 00024400 L_out=1.509 Acc=0.82 ---181597.12 ms/batch
Step 00024450 L_out=0.096 Acc=0.87 ---205028.54 ms/batch
Step 00024500 L_out=0.381 Acc=0.90 ---228336.74 ms/batch
Step 00024550 L_out=0.237 Acc=0.90 ---251608.59 ms/batch
Step 00024600 L_out=0.673 Acc=0.78 ---275031.29 ms/batch
Step 00024650 L_out=0.339 Acc=0.85 ---298501.99 ms/batch
Round 3 | epoch=6, train costTime=312.0820035934448, | total_activation_sum=20504025
Round 3 | ****EPOCH 7****
Step 00024700 L_out=0.486 Acc=0.80 --- 9782.01 ms/batch
Step 00024750 L_out=0.363 Acc=0.84 ---33260.01 ms/batch
Step 00024800 L_out=0.117 Acc=0.90 ---56611.62 ms/batch
Step 00024850 L_out=0.712 Acc=0.83 ---79971.68 ms/batch
Step 00024900 L_out=0.347 Acc=0.91 ---103379.27 ms/batch
Step 00024950 L_out=0.221 Acc=0.81 ---126826.19 ms/batch
Step 00025000 L_out=0.512 Acc=0.91 ---150323.21 ms/batch
Step 00025050 L_out=0.230 Acc=0.87 ---173906.64 ms/batch
Step 00025100 L_out=0.369 Acc=0.87 ---197419.17 ms/batch
Step 00025150 L_out=1.278 Acc=0.61 ---220604.01 ms/batch
Step 00025200 L_out=0.491 Acc=0.86 ---244007.95 ms/batch
Step 00025250 L_out=0.185 Acc=0.92 ---267373.83 ms/batch
Step 00025300 L_out=0.276 Acc=0.88 ---290656.17 ms/batch
Round 3 | epoch=7, train costTime=312.0684037208557, | total_activation_sum=20439980
Round 3 | ****EPOCH 8****
Step 00025350 L_out=0.147 Acc=0.90 --- 1858.46 ms/batch
Step 00025400 L_out=0.251 Acc=0.93 ---25386.74 ms/batch
Step 00025450 L_out=0.438 Acc=0.88 ---48697.08 ms/batch
Step 00025500 L_out=0.181 Acc=0.83 ---72046.74 ms/batch
Step 00025550 L_out=0.220 Acc=0.92 ---95466.19 ms/batch
Step 00025600 L_out=0.425 Acc=0.73 ---118972.67 ms/batch
Step 00025650 L_out=0.246 Acc=0.89 ---142250.83 ms/batch
Step 00025700 L_out=0.177 Acc=0.92 ---165387.68 ms/batch
Step 00025750 L_out=0.201 Acc=0.83 ---188807.00 ms/batch
Step 00025800 L_out=0.095 Acc=0.94 ---212233.64 ms/batch
Step 00025850 L_out=0.721 Acc=0.83 ---235696.36 ms/batch
Step 00025900 L_out=0.487 Acc=0.90 ---259036.03 ms/batch
Step 00025950 L_out=0.209 Acc=0.93 ---282238.92 ms/batch
Step 00026000 L_out=0.469 Acc=0.90 ---305538.07 ms/batch
Round 3 | epoch=8, train costTime=311.69047713279724, | total_activation_sum=20679578
Round 3 | ****EPOCH 9****
Step 00026050 L_out=0.190 Acc=0.93 ---17458.31 ms/batch
Step 00026100 L_out=0.420 Acc=0.80 ---40734.09 ms/batch
Step 00026150 L_out=0.378 Acc=0.94 ---64305.59 ms/batch
Step 00026200 L_out=0.290 Acc=0.88 ---87882.74 ms/batch
Step 00026250 L_out=0.482 Acc=0.92 ---111297.83 ms/batch
Step 00026300 L_out=0.228 Acc=0.92 ---134765.34 ms/batch
Step 00026350 L_out=0.281 Acc=0.85 ---158066.79 ms/batch
Step 00026400 L_out=0.143 Acc=0.93 ---181379.42 ms/batch
Step 00026450 L_out=0.247 Acc=0.87 ---204650.12 ms/batch
Step 00026500 L_out=0.294 Acc=0.90 ---228185.84 ms/batch
Step 00026550 L_out=0.142 Acc=0.93 ---251435.59 ms/batch
Step 00026600 L_out=0.379 Acc=0.91 ---274781.76 ms/batch
Step 00026650 L_out=0.229 Acc=0.79 ---298009.74 ms/batch
Round 3 | epoch=9, train costTime=312.05253887176514, | total_activation_sum=20627960
Round 3 | ****EPOCH 10****
Step 00026700 L_out=0.738 Acc=0.77 --- 9305.81 ms/batch
Step 00026750 L_out=0.442 Acc=0.93 ---32756.93 ms/batch
Step 00026800 L_out=0.095 Acc=0.95 ---56136.82 ms/batch
Step 00026850 L_out=0.977 Acc=0.91 ---79490.75 ms/batch
Step 00026900 L_out=0.248 Acc=0.91 ---102848.33 ms/batch
Step 00026950 L_out=0.423 Acc=0.91 ---126206.25 ms/batch
Step 00027000 L_out=0.365 Acc=0.90 ---149560.74 ms/batch
Step 00027050 L_out=0.468 Acc=0.78 ---173055.00 ms/batch
Step 00027100 L_out=0.228 Acc=0.92 ---196579.53 ms/batch
Step 00027150 L_out=0.414 Acc=0.92 ---220268.21 ms/batch
Step 00027200 L_out=0.264 Acc=0.89 ---243731.21 ms/batch
Step 00027250 L_out=0.257 Acc=0.96 ---267175.76 ms/batch
Step 00027300 L_out=0.274 Acc=0.90 ---290574.93 ms/batch
Round 3 | epoch=10, train costTime=312.73496079444885, | total_activation_sum=20385279
Round 3 | ****EPOCH 11****
Step 00027350 L_out=0.227 Acc=0.90 --- 1383.48 ms/batch
Step 00027400 L_out=0.227 Acc=0.93 ---24540.42 ms/batch
Step 00027450 L_out=0.294 Acc=0.92 ---47813.23 ms/batch
Step 00027500 L_out=0.261 Acc=0.87 ---71097.09 ms/batch
Step 00027550 L_out=0.245 Acc=0.90 ---94458.70 ms/batch
Step 00027600 L_out=0.295 Acc=0.83 ---117842.65 ms/batch
Step 00027650 L_out=0.480 Acc=0.85 ---141271.54 ms/batch
Step 00027700 L_out=0.168 Acc=0.86 ---164899.52 ms/batch
Step 00027750 L_out=0.163 Acc=0.97 ---188276.21 ms/batch
Step 00027800 L_out=0.400 Acc=0.86 ---211769.90 ms/batch
Step 00027850 L_out=0.242 Acc=0.86 ---235267.41 ms/batch
Step 00027900 L_out=0.391 Acc=0.89 ---258581.65 ms/batch
Step 00027950 L_out=0.126 Acc=0.93 ---281978.93 ms/batch
Step 00028000 L_out=0.207 Acc=0.95 ---305196.30 ms/batch
Round 3 | epoch=11, train costTime=311.6197884082794, | total_activation_sum=20499509
Round 3 | ****EPOCH 12****
Step 00028050 L_out=0.369 Acc=0.93 ---16996.82 ms/batch
Step 00028100 L_out=0.187 Acc=0.91 ---40567.64 ms/batch
Step 00028150 L_out=0.352 Acc=0.88 ---63880.72 ms/batch
Step 00028200 L_out=0.320 Acc=0.93 ---87274.69 ms/batch
Step 00028250 L_out=0.128 Acc=0.95 ---110570.26 ms/batch
Step 00028300 L_out=0.150 Acc=0.92 ---134277.32 ms/batch
Step 00028350 L_out=0.218 Acc=0.89 ---157801.31 ms/batch
Step 00028400 L_out=0.453 Acc=0.82 ---181162.15 ms/batch
Step 00028450 L_out=0.056 Acc=0.92 ---204614.33 ms/batch
Step 00028500 L_out=0.807 Acc=0.83 ---228088.41 ms/batch
Step 00028550 L_out=0.128 Acc=0.94 ---251411.55 ms/batch
Step 00028600 L_out=0.202 Acc=0.90 ---274635.06 ms/batch
Step 00028650 L_out=0.317 Acc=0.89 ---297922.26 ms/batch
Round 3 | epoch=12, train costTime=312.40647172927856, | total_activation_sum=20569632
Round 3 | ****EPOCH 13****
Step 00028700 L_out=0.261 Acc=0.91 --- 8807.13 ms/batch
Step 00028750 L_out=0.136 Acc=0.92 ---32024.45 ms/batch
Step 00028800 L_out=0.180 Acc=0.90 ---55635.72 ms/batch
Step 00028850 L_out=0.098 Acc=0.94 ---78951.92 ms/batch
Step 00028900 L_out=0.216 Acc=0.92 ---102352.30 ms/batch
Step 00028950 L_out=0.136 Acc=0.78 ---125941.62 ms/batch
Step 00029000 L_out=0.172 Acc=0.84 ---149544.83 ms/batch
Step 00029050 L_out=0.332 Acc=0.82 ---173021.42 ms/batch
Step 00029100 L_out=0.142 Acc=0.94 ---196347.96 ms/batch
Step 00029150 L_out=0.122 Acc=0.93 ---219772.22 ms/batch
Step 00029200 L_out=0.400 Acc=0.89 ---243333.56 ms/batch
Step 00029250 L_out=0.320 Acc=0.93 ---266808.81 ms/batch
Step 00029300 L_out=0.385 Acc=0.89 ---290061.19 ms/batch
Round 3 | epoch=13, train costTime=312.582222700119, | total_activation_sum=20600824
Round 3 | ****EPOCH 14****
Step 00029350 L_out=0.198 Acc=0.91 ---  933.53 ms/batch
Step 00029400 L_out=0.325 Acc=0.89 ---24311.02 ms/batch
Step 00029450 L_out=0.216 Acc=0.93 ---47585.90 ms/batch
Step 00029500 L_out=0.208 Acc=0.88 ---70960.22 ms/batch
Step 00029550 L_out=0.301 Acc=0.91 ---94302.98 ms/batch
Step 00029600 L_out=0.261 Acc=0.91 ---117824.06 ms/batch
Step 00029650 L_out=0.211 Acc=0.90 ---141257.48 ms/batch
Step 00029700 L_out=0.198 Acc=0.91 ---164871.20 ms/batch
Step 00029750 L_out=0.284 Acc=0.82 ---188251.11 ms/batch
Step 00029800 L_out=0.185 Acc=0.94 ---211764.21 ms/batch
Step 00029850 L_out=0.462 Acc=0.88 ---235163.41 ms/batch
Step 00029900 L_out=0.104 Acc=0.95 ---258608.49 ms/batch
Step 00029950 L_out=0.128 Acc=0.94 ---281945.24 ms/batch
Step 00030000 L_out=0.136 Acc=0.94 ---305495.19 ms/batch
Round 3 | epoch=14, train costTime=312.57667207717896, | total_activation_sum=20474639
Round 3 | ****EPOCH 15****
Step 00030050 L_out=0.315 Acc=0.90 ---16373.29 ms/batch
Step 00030100 L_out=0.098 Acc=0.94 ---39735.32 ms/batch
Step 00030150 L_out=0.058 Acc=0.96 ---63183.29 ms/batch
Step 00030200 L_out=0.103 Acc=0.94 ---86792.44 ms/batch
Step 00030250 L_out=0.326 Acc=0.86 ---110225.65 ms/batch
Step 00030300 L_out=0.364 Acc=0.88 ---133707.87 ms/batch
Step 00030350 L_out=0.444 Acc=0.86 ---156942.06 ms/batch
Step 00030400 L_out=0.199 Acc=0.90 ---180362.72 ms/batch
Step 00030450 L_out=0.357 Acc=0.89 ---203831.13 ms/batch
Step 00030500 L_out=0.284 Acc=0.91 ---227291.45 ms/batch
Step 00030550 L_out=0.325 Acc=0.84 ---250757.24 ms/batch
Step 00030600 L_out=0.295 Acc=0.89 ---274095.60 ms/batch
Step 00030650 L_out=0.408 Acc=0.92 ---297383.07 ms/batch
Round 3 | epoch=15, train costTime=312.39370608329773, | total_activation_sum=20568202
Round 3 | ****EPOCH 16****
Step 00030700 L_out=0.231 Acc=0.91 --- 8419.40 ms/batch
Step 00030750 L_out=0.174 Acc=0.93 ---31876.56 ms/batch
Step 00030800 L_out=0.132 Acc=0.87 ---55318.75 ms/batch
Step 00030850 L_out=0.508 Acc=0.92 ---78755.19 ms/batch
Step 00030900 L_out=0.222 Acc=0.89 ---101900.09 ms/batch
Step 00030950 L_out=0.569 Acc=0.87 ---125421.01 ms/batch
Step 00031000 L_out=0.517 Acc=0.83 ---148840.07 ms/batch
Step 00031050 L_out=0.218 Acc=0.96 ---172309.17 ms/batch
Step 00031100 L_out=0.176 Acc=0.95 ---195856.69 ms/batch
Step 00031150 L_out=0.358 Acc=0.89 ---219097.00 ms/batch
Step 00031200 L_out=0.490 Acc=0.84 ---242299.68 ms/batch
Step 00031250 L_out=0.181 Acc=0.93 ---265799.04 ms/batch
Step 00031300 L_out=0.219 Acc=0.90 ---289191.99 ms/batch
Round 3 | epoch=16, train costTime=312.18721652030945, | total_activation_sum=20565477
Round 3 | ****EPOCH 17****
Step 00031350 L_out=0.118 Acc=0.93 ---  470.09 ms/batch
Step 00031400 L_out=0.118 Acc=0.92 ---23851.56 ms/batch
Step 00031450 L_out=0.211 Acc=0.95 ---47361.81 ms/batch
Step 00031500 L_out=0.029 Acc=0.94 ---70775.41 ms/batch
Step 00031550 L_out=0.198 Acc=0.97 ---94126.86 ms/batch
Step 00031600 L_out=0.234 Acc=0.93 ---117620.22 ms/batch
Step 00031650 L_out=0.412 Acc=0.89 ---141192.36 ms/batch
Step 00031700 L_out=0.430 Acc=0.88 ---164751.46 ms/batch
Step 00031750 L_out=0.062 Acc=0.95 ---188446.12 ms/batch
Step 00031800 L_out=0.310 Acc=0.89 ---212000.71 ms/batch
Step 00031850 L_out=0.183 Acc=0.94 ---235434.39 ms/batch
Step 00031900 L_out=0.118 Acc=0.97 ---258945.67 ms/batch
Step 00031950 L_out=0.186 Acc=0.95 ---282319.83 ms/batch
Step 00032000 L_out=0.149 Acc=0.90 ---305653.77 ms/batch
Round 3 | epoch=17, train costTime=313.1948254108429, | total_activation_sum=20527303
Round 3 | epoch=18, current m_iou=0.6645575966443731
Round 3 | Best m_IoU is: 0.665, OA is: 0.910 | val costTime=117.81447505950928
Round 3 | ****EPOCH 18****
Step 00032050 L_out=0.220 Acc=0.89 ---15846.78 ms/batch
Step 00032100 L_out=0.083 Acc=0.95 ---39230.56 ms/batch
Step 00032150 L_out=0.293 Acc=0.94 ---62618.51 ms/batch
Step 00032200 L_out=0.373 Acc=0.86 ---85963.01 ms/batch
Step 00032250 L_out=0.202 Acc=0.94 ---109314.13 ms/batch
Step 00032300 L_out=0.114 Acc=0.95 ---132703.20 ms/batch
Step 00032350 L_out=0.295 Acc=0.91 ---156090.09 ms/batch
Step 00032400 L_out=0.039 Acc=0.97 ---179519.46 ms/batch
Step 00032450 L_out=0.350 Acc=0.86 ---202798.54 ms/batch
Step 00032500 L_out=0.151 Acc=0.95 ---226170.00 ms/batch
Step 00032550 L_out=0.106 Acc=0.94 ---249600.80 ms/batch
Step 00032600 L_out=0.134 Acc=0.92 ---273143.52 ms/batch
Step 00032650 L_out=0.131 Acc=0.94 ---296343.43 ms/batch
Round 3 | epoch=18, train costTime=311.74602365493774, | total_activation_sum=20579977
Round 3 | epoch=19, current m_iou=0.6844327136048122
Round 3 | Best m_IoU is: 0.684, OA is: 0.915 | val costTime=117.77928185462952
Round 3 | ****EPOCH 19****
Step 00032700 L_out=0.120 Acc=0.93 --- 8035.47 ms/batch
Step 00032750 L_out=0.210 Acc=0.92 ---31285.30 ms/batch
Step 00032800 L_out=0.229 Acc=0.92 ---54600.44 ms/batch
Step 00032850 L_out=0.261 Acc=0.85 ---78115.78 ms/batch
Step 00032900 L_out=0.549 Acc=0.85 ---101582.92 ms/batch
Step 00032950 L_out=0.083 Acc=0.94 ---125031.52 ms/batch
Step 00033000 L_out=0.577 Acc=0.88 ---148315.86 ms/batch
Step 00033050 L_out=0.247 Acc=0.96 ---171720.50 ms/batch
Step 00033100 L_out=0.059 Acc=0.95 ---195153.20 ms/batch
Step 00033150 L_out=0.574 Acc=0.84 ---218504.02 ms/batch
Step 00033200 L_out=0.081 Acc=0.94 ---242118.11 ms/batch
Step 00033250 L_out=0.111 Acc=0.93 ---265556.08 ms/batch
Step 00033300 L_out=0.534 Acc=0.69 ---288848.50 ms/batch
Step 00033350 L_out=0.061 Acc=0.91 ---312237.03 ms/batch
Round 3 | epoch=19, train costTime=312.2374653816223, | total_activation_sum=20521000
Round 3 | epoch=20, current m_iou=0.6884514661889922
Round 3 | Best m_IoU is: 0.688, OA is: 0.919 | val costTime=118.08617091178894
Round 3 | ****EPOCH 20****
Step 00033400 L_out=0.275 Acc=0.93 ---23145.21 ms/batch
Step 00033450 L_out=0.057 Acc=0.97 ---46693.49 ms/batch
Step 00033500 L_out=0.236 Acc=0.81 ---70143.60 ms/batch
Step 00033550 L_out=0.202 Acc=0.88 ---93510.21 ms/batch
Step 00033600 L_out=0.489 Acc=0.79 ---117030.54 ms/batch
Step 00033650 L_out=0.061 Acc=0.91 ---140384.38 ms/batch
Step 00033700 L_out=0.208 Acc=0.95 ---163741.72 ms/batch
Step 00033750 L_out=0.144 Acc=0.95 ---187087.04 ms/batch
Step 00033800 L_out=0.140 Acc=0.95 ---210468.01 ms/batch
Step 00033850 L_out=0.365 Acc=0.87 ---233580.60 ms/batch
Step 00033900 L_out=0.080 Acc=0.93 ---257029.76 ms/batch
Step 00033950 L_out=0.281 Acc=0.94 ---280354.24 ms/batch
Step 00034000 L_out=0.201 Acc=0.90 ---303738.24 ms/batch
Round 3 | epoch=20, train costTime=311.7386631965637, | total_activation_sum=20583608
Round 3 | epoch=21, current m_iou=0.7073242396519965
Round 3 | Best m_IoU is: 0.707, OA is: 0.926 | val costTime=118.32348871231079
Round 3 | ****EPOCH 21****
Step 00034050 L_out=0.173 Acc=0.95 ---15470.61 ms/batch
Step 00034100 L_out=0.175 Acc=0.94 ---38948.88 ms/batch
Step 00034150 L_out=0.304 Acc=0.82 ---62415.82 ms/batch
Step 00034200 L_out=0.111 Acc=0.96 ---85823.60 ms/batch
Step 00034250 L_out=0.119 Acc=0.91 ---109235.61 ms/batch
Step 00034300 L_out=0.140 Acc=0.89 ---132528.82 ms/batch
Step 00034350 L_out=0.366 Acc=0.94 ---155892.59 ms/batch
Step 00034400 L_out=0.130 Acc=0.97 ---179339.86 ms/batch
Step 00034450 L_out=0.162 Acc=0.94 ---202785.91 ms/batch
Step 00034500 L_out=0.055 Acc=0.96 ---226262.83 ms/batch
Step 00034550 L_out=0.257 Acc=0.92 ---249656.83 ms/batch
Step 00034600 L_out=0.098 Acc=0.97 ---273088.52 ms/batch
Step 00034650 L_out=0.201 Acc=0.93 ---296692.86 ms/batch
Round 3 | epoch=21, train costTime=312.5216906070709, | total_activation_sum=20454831
Round 3 | epoch=22, current m_iou=0.7126856900726637
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.99416875839233
Round 3 | ****EPOCH 22****
Step 00034700 L_out=0.240 Acc=0.89 --- 7656.41 ms/batch
Step 00034750 L_out=0.166 Acc=0.93 ---30991.75 ms/batch
Step 00034800 L_out=0.501 Acc=0.90 ---54540.94 ms/batch
Step 00034850 L_out=0.067 Acc=0.93 ---77991.38 ms/batch
Step 00034900 L_out=0.351 Acc=0.88 ---101442.64 ms/batch
Step 00034950 L_out=0.260 Acc=0.95 ---124885.42 ms/batch
Step 00035000 L_out=0.176 Acc=0.95 ---148362.37 ms/batch
Step 00035050 L_out=0.139 Acc=0.96 ---171809.16 ms/batch
Step 00035100 L_out=0.210 Acc=0.94 ---195256.66 ms/batch
Step 00035150 L_out=0.230 Acc=0.95 ---218603.10 ms/batch
Step 00035200 L_out=3.152 Acc=0.62 ---241850.19 ms/batch
Step 00035250 L_out=0.116 Acc=0.95 ---265284.03 ms/batch
Step 00035300 L_out=0.231 Acc=0.95 ---288713.13 ms/batch
Step 00035350 L_out=0.227 Acc=0.84 ---312164.09 ms/batch
Round 3 | epoch=22, train costTime=312.61685371398926, | total_activation_sum=20605764
Round 3 | epoch=23, current m_iou=0.708128367829915
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.7172360420227
Round 3 | ****EPOCH 23****
Step 00035400 L_out=0.192 Acc=0.95 ---22760.53 ms/batch
Step 00035450 L_out=0.199 Acc=0.94 ---46145.13 ms/batch
Step 00035500 L_out=0.280 Acc=0.94 ---69411.36 ms/batch
Step 00035550 L_out=0.087 Acc=0.94 ---92679.82 ms/batch
Step 00035600 L_out=0.148 Acc=0.93 ---116073.54 ms/batch
Step 00035650 L_out=0.152 Acc=0.91 ---139417.88 ms/batch
Step 00035700 L_out=0.236 Acc=0.87 ---162675.11 ms/batch
Step 00035750 L_out=0.275 Acc=0.91 ---186189.74 ms/batch
Step 00035800 L_out=0.111 Acc=0.94 ---209723.83 ms/batch
Step 00035850 L_out=0.248 Acc=0.94 ---233146.82 ms/batch
Step 00035900 L_out=0.672 Acc=0.74 ---256535.67 ms/batch
Step 00035950 L_out=0.155 Acc=0.91 ---279973.05 ms/batch
Step 00036000 L_out=0.569 Acc=0.90 ---303589.17 ms/batch
Round 3 | epoch=23, train costTime=311.95940256118774, | total_activation_sum=20721963
Round 3 | epoch=24, current m_iou=0.6998725741099748
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.83954811096191
Round 3 | ****EPOCH 24****
Step 00036050 L_out=0.399 Acc=0.89 ---14979.64 ms/batch
Step 00036100 L_out=0.121 Acc=0.92 ---38601.93 ms/batch
Step 00036150 L_out=0.060 Acc=0.97 ---62049.76 ms/batch
Step 00036200 L_out=0.120 Acc=0.92 ---85412.61 ms/batch
Step 00036250 L_out=0.105 Acc=0.96 ---108885.49 ms/batch
Step 00036300 L_out=0.293 Acc=0.92 ---132135.84 ms/batch
Step 00036350 L_out=0.135 Acc=0.93 ---155335.73 ms/batch
Step 00036400 L_out=0.134 Acc=0.93 ---178793.01 ms/batch
Step 00036450 L_out=0.357 Acc=0.80 ---202206.85 ms/batch
Step 00036500 L_out=0.235 Acc=0.87 ---225589.09 ms/batch
Step 00036550 L_out=0.301 Acc=0.92 ---248940.13 ms/batch
Step 00036600 L_out=0.098 Acc=0.91 ---272438.86 ms/batch
Step 00036650 L_out=0.141 Acc=0.96 ---295914.23 ms/batch
Round 3 | epoch=24, train costTime=312.241370677948, | total_activation_sum=20445324
Round 3 | epoch=25, current m_iou=0.7028651497591364
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=118.1958224773407
Round 3 | ****EPOCH 25****
Step 00036700 L_out=0.126 Acc=0.92 --- 7119.07 ms/batch
Step 00036750 L_out=0.166 Acc=0.93 ---30545.54 ms/batch
Step 00036800 L_out=0.106 Acc=0.94 ---54084.58 ms/batch
Step 00036850 L_out=0.051 Acc=0.97 ---77551.80 ms/batch
Step 00036900 L_out=0.109 Acc=0.93 ---100841.26 ms/batch
Step 00036950 L_out=0.214 Acc=0.91 ---124309.18 ms/batch
Step 00037000 L_out=0.155 Acc=0.83 ---147515.43 ms/batch
Step 00037050 L_out=0.102 Acc=0.94 ---170746.46 ms/batch
Step 00037100 L_out=0.724 Acc=0.82 ---194096.11 ms/batch
Step 00037150 L_out=0.300 Acc=0.92 ---217579.69 ms/batch
Step 00037200 L_out=0.346 Acc=0.87 ---241050.50 ms/batch
Step 00037250 L_out=0.109 Acc=0.97 ---264526.23 ms/batch
Step 00037300 L_out=0.352 Acc=0.91 ---287996.56 ms/batch
Step 00037350 L_out=0.293 Acc=0.95 ---311419.35 ms/batch
Round 3 | epoch=25, train costTime=312.3565671443939, | total_activation_sum=20500576
Round 3 | epoch=26, current m_iou=0.7045520394297773
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.59875726699829
Round 3 | ****EPOCH 26****
Step 00037400 L_out=0.085 Acc=0.93 ---22422.03 ms/batch
Step 00037450 L_out=0.122 Acc=0.89 ---45682.33 ms/batch
Step 00037500 L_out=0.100 Acc=0.95 ---68970.92 ms/batch
Step 00037550 L_out=0.098 Acc=0.95 ---92306.64 ms/batch
Step 00037600 L_out=0.180 Acc=0.85 ---115936.46 ms/batch
Step 00037650 L_out=0.183 Acc=0.94 ---139278.11 ms/batch
Step 00037700 L_out=0.086 Acc=0.95 ---162616.14 ms/batch
Step 00037750 L_out=0.303 Acc=0.92 ---185972.92 ms/batch
Step 00037800 L_out=0.068 Acc=0.92 ---209477.12 ms/batch
Step 00037850 L_out=0.275 Acc=0.89 ---232827.99 ms/batch
Step 00037900 L_out=0.098 Acc=0.98 ---256480.30 ms/batch
Step 00037950 L_out=0.107 Acc=0.93 ---279784.82 ms/batch
Step 00038000 L_out=0.159 Acc=0.96 ---303270.18 ms/batch
Round 3 | epoch=26, train costTime=312.0631949901581, | total_activation_sum=20469955
Round 3 | epoch=27, current m_iou=0.7032023024468637
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.9529812335968
Round 3 | ****EPOCH 27****
Step 00038050 L_out=0.161 Acc=0.93 ---14572.45 ms/batch
Step 00038100 L_out=0.520 Acc=0.88 ---38125.36 ms/batch
Step 00038150 L_out=0.074 Acc=0.94 ---61460.97 ms/batch
Step 00038200 L_out=0.113 Acc=0.95 ---84919.18 ms/batch
Step 00038250 L_out=0.147 Acc=0.92 ---108468.78 ms/batch
Step 00038300 L_out=0.188 Acc=0.95 ---131725.09 ms/batch
Step 00038350 L_out=0.111 Acc=0.93 ---155063.44 ms/batch
Step 00038400 L_out=0.141 Acc=0.95 ---178436.96 ms/batch
Step 00038450 L_out=1.185 Acc=0.79 ---201864.75 ms/batch
Step 00038500 L_out=0.147 Acc=0.93 ---225243.01 ms/batch
Step 00038550 L_out=0.399 Acc=0.82 ---248542.01 ms/batch
Step 00038600 L_out=0.210 Acc=0.80 ---271768.21 ms/batch
Step 00038650 L_out=0.048 Acc=0.93 ---295085.21 ms/batch
Round 3 | epoch=27, train costTime=312.1356427669525, | total_activation_sum=20675306
Round 3 | epoch=28, current m_iou=0.7060805042718371
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.88041019439697
Round 3 | ****EPOCH 28****
Step 00038700 L_out=0.062 Acc=0.94 --- 6587.52 ms/batch
Step 00038750 L_out=0.065 Acc=0.97 ---29680.71 ms/batch
Step 00038800 L_out=0.310 Acc=0.94 ---52999.75 ms/batch
Step 00038850 L_out=0.100 Acc=0.94 ---76294.52 ms/batch
Step 00038900 L_out=0.231 Acc=0.94 ---99737.38 ms/batch
Step 00038950 L_out=0.210 Acc=0.89 ---122900.99 ms/batch
Step 00039000 L_out=0.175 Acc=0.90 ---146424.12 ms/batch
Step 00039050 L_out=0.343 Acc=0.93 ---169827.03 ms/batch
Step 00039100 L_out=0.218 Acc=0.91 ---193399.57 ms/batch
Step 00039150 L_out=0.271 Acc=0.92 ---216755.78 ms/batch
Step 00039200 L_out=0.155 Acc=0.97 ---240256.03 ms/batch
Step 00039250 L_out=0.241 Acc=0.94 ---263536.28 ms/batch
Step 00039300 L_out=0.122 Acc=0.96 ---286893.97 ms/batch
Step 00039350 L_out=0.095 Acc=0.96 ---310325.76 ms/batch
Round 3 | epoch=28, train costTime=311.69179034233093, | total_activation_sum=20453625
Round 3 | epoch=29, current m_iou=0.706590276934685
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.47830653190613
Round 3 | ****EPOCH 29****
Step 00039400 L_out=0.313 Acc=0.90 ---22090.38 ms/batch
Step 00039450 L_out=0.223 Acc=0.93 ---45438.12 ms/batch
Step 00039500 L_out=0.117 Acc=0.96 ---68816.98 ms/batch
Step 00039550 L_out=0.096 Acc=0.97 ---92428.12 ms/batch
Step 00039600 L_out=0.070 Acc=0.94 ---115642.69 ms/batch
Step 00039650 L_out=0.121 Acc=0.91 ---138904.04 ms/batch
Step 00039700 L_out=0.043 Acc=0.93 ---162211.37 ms/batch
Step 00039750 L_out=0.072 Acc=0.94 ---185497.58 ms/batch
Step 00039800 L_out=0.324 Acc=0.94 ---208839.86 ms/batch
Step 00039850 L_out=0.156 Acc=0.86 ---232363.31 ms/batch
Step 00039900 L_out=0.174 Acc=0.94 ---255577.75 ms/batch
Step 00039950 L_out=0.099 Acc=0.92 ---279074.73 ms/batch
Step 00040000 L_out=0.144 Acc=0.94 ---302445.29 ms/batch
Round 3 | epoch=29, train costTime=311.73449754714966, | total_activation_sum=20438288
Round 3 | epoch=30, current m_iou=0.7081371749383675
Round 3 | Best m_IoU is: 0.713, OA is: 0.928 | val costTime=117.68322157859802
Round 3 | ****EPOCH 30****
Round 4 | ****EPOCH 0****
Step 00040050 L_out=3.030 Acc=0.61 ---14069.74 ms/batch
Step 00040100 L_out=1.182 Acc=0.66 ---37519.18 ms/batch
Step 00040150 L_out=1.941 Acc=0.68 ---61015.20 ms/batch
Step 00040200 L_out=1.105 Acc=0.73 ---84523.79 ms/batch
Step 00040250 L_out=1.259 Acc=0.75 ---108004.98 ms/batch
Step 00040300 L_out=1.873 Acc=0.68 ---131393.47 ms/batch
Step 00040350 L_out=1.172 Acc=0.64 ---155062.53 ms/batch
Step 00040400 L_out=0.394 Acc=0.88 ---178737.23 ms/batch
Step 00040450 L_out=1.271 Acc=0.79 ---202404.04 ms/batch
Step 00040500 L_out=1.082 Acc=0.87 ---225947.25 ms/batch
Step 00040550 L_out=0.295 Acc=0.89 ---249488.20 ms/batch
Step 00040600 L_out=0.679 Acc=0.90 ---273084.23 ms/batch
Step 00040650 L_out=0.567 Acc=0.91 ---296424.66 ms/batch
Round 4 | epoch=0, train costTime=313.8512942790985, | total_activation_sum=28077473
Round 4 | ****EPOCH 1****
Step 00040700 L_out=1.348 Acc=0.65 --- 6013.72 ms/batch
Step 00040750 L_out=0.577 Acc=0.89 ---29395.88 ms/batch
Step 00040800 L_out=0.513 Acc=0.87 ---52914.61 ms/batch
Step 00040850 L_out=0.524 Acc=0.88 ---76135.85 ms/batch
Step 00040900 L_out=0.399 Acc=0.84 ---99410.50 ms/batch
Step 00040950 L_out=0.642 Acc=0.80 ---122962.99 ms/batch
Step 00041000 L_out=0.847 Acc=0.83 ---146158.45 ms/batch
Step 00041050 L_out=0.381 Acc=0.88 ---169582.32 ms/batch
Step 00041100 L_out=0.645 Acc=0.87 ---193334.07 ms/batch
Step 00041150 L_out=0.379 Acc=0.85 ---216716.46 ms/batch
Step 00041200 L_out=0.402 Acc=0.94 ---240088.61 ms/batch
Step 00041250 L_out=0.207 Acc=0.88 ---263540.20 ms/batch
Step 00041300 L_out=0.884 Acc=0.60 ---286965.55 ms/batch
Step 00041350 L_out=0.368 Acc=0.94 ---310313.09 ms/batch
Round 4 | epoch=1, train costTime=312.1576910018921, | total_activation_sum=26657105
Round 4 | ****EPOCH 2****
Step 00041400 L_out=0.559 Acc=0.88 ---21553.60 ms/batch
Step 00041450 L_out=0.358 Acc=0.86 ---44938.31 ms/batch
Step 00041500 L_out=0.728 Acc=0.92 ---68245.24 ms/batch
Step 00041550 L_out=0.666 Acc=0.92 ---91522.06 ms/batch
Step 00041600 L_out=0.516 Acc=0.87 ---115086.34 ms/batch
Step 00041650 L_out=0.309 Acc=0.91 ---138517.20 ms/batch
Step 00041700 L_out=3.273 Acc=0.72 ---161850.38 ms/batch
Step 00041750 L_out=0.641 Acc=0.92 ---185060.18 ms/batch
Step 00041800 L_out=0.486 Acc=0.90 ---208374.21 ms/batch
Step 00041850 L_out=0.816 Acc=0.94 ---232045.99 ms/batch
Step 00041900 L_out=0.198 Acc=0.89 ---255588.20 ms/batch
Step 00041950 L_out=0.334 Acc=0.95 ---279228.74 ms/batch
Step 00042000 L_out=0.207 Acc=0.88 ---302710.83 ms/batch
Round 4 | epoch=2, train costTime=312.5263240337372, | total_activation_sum=26626613
Round 4 | ****EPOCH 3****
Step 00042050 L_out=0.494 Acc=0.92 ---13741.36 ms/batch
Step 00042100 L_out=0.575 Acc=0.90 ---37045.54 ms/batch
Step 00042150 L_out=0.291 Acc=0.89 ---60524.88 ms/batch
Step 00042200 L_out=0.874 Acc=0.86 ---84046.59 ms/batch
Step 00042250 L_out=0.301 Acc=0.84 ---107275.25 ms/batch
Step 00042300 L_out=0.286 Acc=0.92 ---130758.60 ms/batch
Step 00042350 L_out=0.951 Acc=0.84 ---154471.73 ms/batch
Step 00042400 L_out=0.160 Acc=0.93 ---177870.64 ms/batch
Step 00042450 L_out=1.465 Acc=0.66 ---201364.56 ms/batch
Step 00042500 L_out=0.304 Acc=0.91 ---224729.05 ms/batch
Step 00042550 L_out=0.689 Acc=0.88 ---248022.32 ms/batch
Step 00042600 L_out=0.302 Acc=0.88 ---271431.07 ms/batch
Step 00042650 L_out=0.302 Acc=0.93 ---294912.60 ms/batch
Round 4 | epoch=3, train costTime=312.8058559894562, | total_activation_sum=26773962
Round 4 | ****EPOCH 4****
Step 00042700 L_out=0.432 Acc=0.88 --- 5638.04 ms/batch
Step 00042750 L_out=1.216 Acc=0.75 ---29061.20 ms/batch
Step 00042800 L_out=0.332 Acc=0.84 ---52246.76 ms/batch
Step 00042850 L_out=0.664 Acc=0.84 ---75488.91 ms/batch
Step 00042900 L_out=0.410 Acc=0.87 ---99001.72 ms/batch
Step 00042950 L_out=0.391 Acc=0.93 ---122569.16 ms/batch
Step 00043000 L_out=1.753 Acc=0.86 ---145823.59 ms/batch
Step 00043050 L_out=0.594 Acc=0.77 ---169003.73 ms/batch
Step 00043100 L_out=0.338 Acc=0.90 ---192337.88 ms/batch
Step 00043150 L_out=0.403 Acc=0.77 ---215882.99 ms/batch
Step 00043200 L_out=0.370 Acc=0.90 ---239275.82 ms/batch
Step 00043250 L_out=0.677 Acc=0.95 ---262926.74 ms/batch
Step 00043300 L_out=0.341 Acc=0.92 ---286190.53 ms/batch
Step 00043350 L_out=0.841 Acc=0.86 ---309481.07 ms/batch
Round 4 | epoch=4, train costTime=311.81753730773926, | total_activation_sum=26643326
Round 4 | ****EPOCH 5****
Step 00043400 L_out=0.270 Acc=0.89 ---20838.57 ms/batch
Step 00043450 L_out=0.224 Acc=0.94 ---44378.80 ms/batch
Step 00043500 L_out=0.236 Acc=0.95 ---67792.10 ms/batch
Step 00043550 L_out=0.287 Acc=0.94 ---91229.09 ms/batch
Step 00043600 L_out=0.364 Acc=0.93 ---114357.85 ms/batch
Step 00043650 L_out=0.200 Acc=0.95 ---137809.23 ms/batch
Step 00043700 L_out=0.226 Acc=0.90 ---161419.81 ms/batch
Step 00043750 L_out=0.492 Acc=0.93 ---184951.73 ms/batch
Step 00043800 L_out=0.251 Acc=0.90 ---208760.69 ms/batch
Step 00043850 L_out=0.083 Acc=0.89 ---232069.16 ms/batch
Step 00043900 L_out=0.212 Acc=0.93 ---255573.31 ms/batch
Step 00043950 L_out=0.224 Acc=0.90 ---278823.32 ms/batch
Step 00044000 L_out=0.297 Acc=0.90 ---302077.00 ms/batch
Round 4 | epoch=5, train costTime=312.4722697734833, | total_activation_sum=26649874
Round 4 | ****EPOCH 6****
Step 00044050 L_out=0.289 Acc=0.86 ---13151.30 ms/batch
Step 00044100 L_out=0.306 Acc=0.90 ---36724.92 ms/batch
Step 00044150 L_out=0.165 Acc=0.93 ---60050.09 ms/batch
Step 00044200 L_out=0.455 Acc=0.89 ---83515.02 ms/batch
Step 00044250 L_out=0.313 Acc=0.86 ---106867.79 ms/batch
Step 00044300 L_out=1.733 Acc=0.72 ---130394.14 ms/batch
Step 00044350 L_out=0.369 Acc=0.92 ---153897.80 ms/batch
Step 00044400 L_out=0.132 Acc=0.94 ---177080.51 ms/batch
Step 00044450 L_out=0.096 Acc=0.98 ---200519.12 ms/batch
Step 00044500 L_out=0.160 Acc=0.94 ---224103.24 ms/batch
Step 00044550 L_out=0.270 Acc=0.92 ---247419.95 ms/batch
Step 00044600 L_out=0.449 Acc=0.79 ---270826.26 ms/batch
Step 00044650 L_out=0.371 Acc=0.95 ---294272.55 ms/batch
Round 4 | epoch=6, train costTime=312.4574365615845, | total_activation_sum=26686976
Round 4 | ****EPOCH 7****
Step 00044700 L_out=0.230 Acc=0.89 --- 5182.46 ms/batch
Step 00044750 L_out=0.399 Acc=0.88 ---28399.92 ms/batch
Step 00044800 L_out=0.480 Acc=0.96 ---51875.45 ms/batch
Step 00044850 L_out=0.087 Acc=0.96 ---75428.34 ms/batch
Step 00044900 L_out=0.729 Acc=0.84 ---98815.04 ms/batch
Step 00044950 L_out=0.306 Acc=0.88 ---122287.79 ms/batch
Step 00045000 L_out=0.127 Acc=0.95 ---145428.27 ms/batch
Step 00045050 L_out=0.351 Acc=0.92 ---168844.22 ms/batch
Step 00045100 L_out=0.223 Acc=0.96 ---192236.51 ms/batch
Step 00045150 L_out=0.098 Acc=0.94 ---215585.82 ms/batch
Step 00045200 L_out=0.128 Acc=0.94 ---239152.67 ms/batch
Step 00045250 L_out=0.532 Acc=0.86 ---262530.79 ms/batch
Step 00045300 L_out=0.338 Acc=0.92 ---285868.90 ms/batch
Step 00045350 L_out=0.089 Acc=0.92 ---309157.84 ms/batch
Round 4 | epoch=7, train costTime=311.96587586402893, | total_activation_sum=26563923
Round 4 | ****EPOCH 8****
Step 00045400 L_out=0.479 Acc=0.86 ---20487.51 ms/batch
Step 00045450 L_out=0.212 Acc=0.91 ---44115.55 ms/batch
Step 00045500 L_out=0.076 Acc=0.95 ---67657.62 ms/batch
Step 00045550 L_out=0.606 Acc=0.90 ---90947.32 ms/batch
Step 00045600 L_out=0.252 Acc=0.80 ---114213.59 ms/batch
Step 00045650 L_out=0.576 Acc=0.89 ---137863.44 ms/batch
Step 00045700 L_out=0.559 Acc=0.85 ---160959.53 ms/batch
Step 00045750 L_out=0.770 Acc=0.72 ---184347.84 ms/batch
Step 00045800 L_out=0.273 Acc=0.90 ---207744.83 ms/batch
Step 00045850 L_out=0.521 Acc=0.85 ---231175.86 ms/batch
Step 00045900 L_out=0.131 Acc=0.85 ---254481.27 ms/batch
Step 00045950 L_out=0.214 Acc=0.95 ---277746.58 ms/batch
Step 00046000 L_out=0.303 Acc=0.92 ---301158.23 ms/batch
Round 4 | epoch=8, train costTime=311.91475677490234, | total_activation_sum=26864680
Round 4 | ****EPOCH 9****
Step 00046050 L_out=0.225 Acc=0.94 ---12481.20 ms/batch
Step 00046100 L_out=0.619 Acc=0.91 ---35735.63 ms/batch
Step 00046150 L_out=0.151 Acc=0.95 ---58909.71 ms/batch
Step 00046200 L_out=0.382 Acc=0.92 ---82291.08 ms/batch
Step 00046250 L_out=0.193 Acc=0.92 ---105602.68 ms/batch
Step 00046300 L_out=0.260 Acc=0.93 ---128837.51 ms/batch
Step 00046350 L_out=0.324 Acc=0.93 ---152181.53 ms/batch
Step 00046400 L_out=0.275 Acc=0.96 ---175762.28 ms/batch
Step 00046450 L_out=0.102 Acc=0.97 ---199395.84 ms/batch
Step 00046500 L_out=0.373 Acc=0.84 ---222535.89 ms/batch
Step 00046550 L_out=0.225 Acc=0.90 ---245763.91 ms/batch
Step 00046600 L_out=0.259 Acc=0.95 ---269269.91 ms/batch
Step 00046650 L_out=0.340 Acc=0.91 ---292566.66 ms/batch
Round 4 | epoch=9, train costTime=311.40292954444885, | total_activation_sum=26518625
Round 4 | ****EPOCH 10****
Step 00046700 L_out=0.627 Acc=0.89 --- 4651.71 ms/batch
Step 00046750 L_out=0.550 Acc=0.91 ---27864.63 ms/batch
Step 00046800 L_out=0.280 Acc=0.89 ---51505.19 ms/batch
Step 00046850 L_out=0.090 Acc=0.98 ---74991.02 ms/batch
Step 00046900 L_out=0.298 Acc=0.92 ---98410.79 ms/batch
Step 00046950 L_out=0.323 Acc=0.90 ---121755.12 ms/batch
Step 00047000 L_out=0.134 Acc=0.96 ---145042.09 ms/batch
Step 00047050 L_out=0.217 Acc=0.95 ---168634.64 ms/batch
Step 00047100 L_out=1.479 Acc=0.71 ---192029.41 ms/batch
Step 00047150 L_out=0.244 Acc=0.93 ---215492.25 ms/batch
Step 00047200 L_out=0.108 Acc=0.95 ---238817.32 ms/batch
Step 00047250 L_out=0.666 Acc=0.91 ---262005.31 ms/batch
Step 00047300 L_out=0.357 Acc=0.94 ---285533.89 ms/batch
Step 00047350 L_out=0.253 Acc=0.92 ---308962.44 ms/batch
Round 4 | epoch=10, train costTime=312.2819426059723, | total_activation_sum=26671732
Round 4 | ****EPOCH 11****
Step 00047400 L_out=0.385 Acc=0.91 ---20185.02 ms/batch
Step 00047450 L_out=0.080 Acc=0.96 ---43733.08 ms/batch
Step 00047500 L_out=0.218 Acc=0.95 ---67217.00 ms/batch
Step 00047550 L_out=0.302 Acc=0.94 ---90759.37 ms/batch
Step 00047600 L_out=0.060 Acc=0.97 ---114315.29 ms/batch
Step 00047650 L_out=0.396 Acc=0.79 ---137649.12 ms/batch
Step 00047700 L_out=0.335 Acc=0.88 ---161051.52 ms/batch
Step 00047750 L_out=0.182 Acc=0.95 ---184389.94 ms/batch
Step 00047800 L_out=0.082 Acc=0.95 ---207755.82 ms/batch
Step 00047850 L_out=0.367 Acc=0.94 ---231252.84 ms/batch
Step 00047900 L_out=0.299 Acc=0.92 ---254414.46 ms/batch
Step 00047950 L_out=0.156 Acc=0.86 ---277835.30 ms/batch
Step 00048000 L_out=0.157 Acc=0.92 ---301355.34 ms/batch
Round 4 | epoch=11, train costTime=312.58945870399475, | total_activation_sum=26758931
Round 4 | ****EPOCH 12****
Step 00048050 L_out=0.488 Acc=0.88 ---12107.70 ms/batch
Step 00048100 L_out=0.227 Acc=0.96 ---35445.26 ms/batch
Step 00048150 L_out=0.109 Acc=0.93 ---59064.10 ms/batch
Step 00048200 L_out=0.616 Acc=0.83 ---82729.75 ms/batch
Step 00048250 L_out=0.358 Acc=0.96 ---106296.07 ms/batch
Step 00048300 L_out=0.261 Acc=0.95 ---129550.53 ms/batch
Step 00048350 L_out=0.177 Acc=0.95 ---152722.07 ms/batch
Step 00048400 L_out=0.063 Acc=0.95 ---176030.71 ms/batch
Step 00048450 L_out=0.160 Acc=0.95 ---199181.46 ms/batch
Step 00048500 L_out=0.184 Acc=0.93 ---222627.00 ms/batch
Step 00048550 L_out=0.195 Acc=0.93 ---246148.38 ms/batch
Step 00048600 L_out=0.145 Acc=0.94 ---269613.99 ms/batch
Step 00048650 L_out=0.182 Acc=0.92 ---293060.24 ms/batch
Round 4 | epoch=12, train costTime=312.30713415145874, | total_activation_sum=26618247
Round 4 | ****EPOCH 13****
Step 00048700 L_out=0.051 Acc=0.96 --- 4292.08 ms/batch
Step 00048750 L_out=0.305 Acc=0.91 ---27719.05 ms/batch
Step 00048800 L_out=0.241 Acc=0.96 ---51223.80 ms/batch
Step 00048850 L_out=0.176 Acc=0.94 ---74768.78 ms/batch
Step 00048900 L_out=0.445 Acc=0.87 ---98142.18 ms/batch
Step 00048950 L_out=0.247 Acc=0.91 ---121349.47 ms/batch
Step 00049000 L_out=0.248 Acc=0.93 ---144838.84 ms/batch
Step 00049050 L_out=0.164 Acc=0.97 ---168169.36 ms/batch
Step 00049100 L_out=0.233 Acc=0.94 ---191489.81 ms/batch
Step 00049150 L_out=0.307 Acc=0.94 ---214829.42 ms/batch
Step 00049200 L_out=0.196 Acc=0.92 ---238328.59 ms/batch
Step 00049250 L_out=0.173 Acc=0.96 ---261648.56 ms/batch
Step 00049300 L_out=0.571 Acc=0.86 ---284941.67 ms/batch
Step 00049350 L_out=0.099 Acc=0.96 ---308219.26 ms/batch
Round 4 | epoch=13, train costTime=311.950560092926, | total_activation_sum=26840986
Round 4 | ****EPOCH 14****
Step 00049400 L_out=0.259 Acc=0.93 ---19659.14 ms/batch
Step 00049450 L_out=0.199 Acc=0.96 ---43020.63 ms/batch
Step 00049500 L_out=0.244 Acc=0.93 ---66498.53 ms/batch
Step 00049550 L_out=0.144 Acc=0.95 ---89925.35 ms/batch
Step 00049600 L_out=0.220 Acc=0.90 ---113239.45 ms/batch
Step 00049650 L_out=0.232 Acc=0.92 ---136584.49 ms/batch
Step 00049700 L_out=0.120 Acc=0.96 ---160027.70 ms/batch
Step 00049750 L_out=0.160 Acc=0.93 ---183452.89 ms/batch
Step 00049800 L_out=0.097 Acc=0.97 ---207119.24 ms/batch
Step 00049850 L_out=0.157 Acc=0.82 ---230269.05 ms/batch
Step 00049900 L_out=0.088 Acc=0.96 ---253780.42 ms/batch
Step 00049950 L_out=0.394 Acc=0.88 ---276908.85 ms/batch
Step 00050000 L_out=0.101 Acc=0.92 ---300354.74 ms/batch
Round 4 | epoch=14, train costTime=312.153511762619, | total_activation_sum=26699937
Round 4 | ****EPOCH 15****
Step 00050050 L_out=0.298 Acc=0.92 ---11750.91 ms/batch
Step 00050100 L_out=0.224 Acc=0.90 ---35166.79 ms/batch
Step 00050150 L_out=0.258 Acc=0.91 ---58688.85 ms/batch
Step 00050200 L_out=0.291 Acc=0.93 ---82062.62 ms/batch
Step 00050250 L_out=0.335 Acc=0.90 ---105584.50 ms/batch
Step 00050300 L_out=0.183 Acc=0.93 ---129024.25 ms/batch
Step 00050350 L_out=0.265 Acc=0.95 ---152642.54 ms/batch
Step 00050400 L_out=0.131 Acc=0.93 ---176231.95 ms/batch
Step 00050450 L_out=0.115 Acc=0.94 ---199740.22 ms/batch
Step 00050500 L_out=0.092 Acc=0.94 ---222937.64 ms/batch
Step 00050550 L_out=0.386 Acc=0.90 ---246273.98 ms/batch
Step 00050600 L_out=0.179 Acc=0.94 ---269552.71 ms/batch
Step 00050650 L_out=0.541 Acc=0.80 ---292958.96 ms/batch
Round 4 | epoch=15, train costTime=312.5655286312103, | total_activation_sum=26810900
Round 4 | ****EPOCH 16****
Step 00050700 L_out=0.293 Acc=0.94 --- 3675.56 ms/batch
Step 00050750 L_out=0.081 Acc=0.96 ---27057.23 ms/batch
Step 00050800 L_out=0.116 Acc=0.95 ---50580.55 ms/batch
Step 00050850 L_out=0.320 Acc=0.88 ---73893.85 ms/batch
Step 00050900 L_out=0.294 Acc=0.94 ---97249.72 ms/batch
Step 00050950 L_out=0.354 Acc=0.96 ---120731.66 ms/batch
Step 00051000 L_out=0.127 Acc=0.95 ---144440.33 ms/batch
Step 00051050 L_out=0.111 Acc=0.94 ---167980.64 ms/batch
Step 00051100 L_out=0.196 Acc=0.93 ---191325.87 ms/batch
Step 00051150 L_out=0.089 Acc=0.94 ---214715.25 ms/batch
Step 00051200 L_out=0.140 Acc=0.95 ---238211.65 ms/batch
Step 00051250 L_out=0.421 Acc=0.95 ---261728.10 ms/batch
Step 00051300 L_out=0.185 Acc=0.94 ---285187.93 ms/batch
Step 00051350 L_out=0.057 Acc=0.95 ---308606.89 ms/batch
Round 4 | epoch=16, train costTime=312.78792452812195, | total_activation_sum=26566613
Round 4 | ****EPOCH 17****
Step 00051400 L_out=0.345 Acc=0.92 ---19102.30 ms/batch
Step 00051450 L_out=0.305 Acc=0.90 ---42578.82 ms/batch
Step 00051500 L_out=0.354 Acc=0.92 ---65895.85 ms/batch
Step 00051550 L_out=0.094 Acc=0.97 ---89288.68 ms/batch
Step 00051600 L_out=0.382 Acc=0.88 ---112606.84 ms/batch
Step 00051650 L_out=0.105 Acc=0.96 ---136036.90 ms/batch
Step 00051700 L_out=0.056 Acc=0.99 ---159253.79 ms/batch
Step 00051750 L_out=0.165 Acc=0.97 ---182506.74 ms/batch
Step 00051800 L_out=0.423 Acc=0.91 ---206013.71 ms/batch
Step 00051850 L_out=0.431 Acc=0.87 ---229393.44 ms/batch
Step 00051900 L_out=0.210 Acc=0.93 ---252653.90 ms/batch
Step 00051950 L_out=0.129 Acc=0.95 ---276201.98 ms/batch
Step 00052000 L_out=0.128 Acc=0.94 ---299515.64 ms/batch
Round 4 | epoch=17, train costTime=311.76109051704407, | total_activation_sum=26683812
Round 4 | epoch=18, current m_iou=0.6711043508234588
Round 4 | Best m_IoU is: 0.671, OA is: 0.908 | val costTime=118.09118294715881
Round 4 | ****EPOCH 18****
Step 00052050 L_out=0.154 Acc=0.95 ---11165.27 ms/batch
Step 00052100 L_out=0.276 Acc=0.93 ---34520.95 ms/batch
Step 00052150 L_out=0.203 Acc=0.93 ---57916.30 ms/batch
Step 00052200 L_out=0.157 Acc=0.98 ---81469.84 ms/batch
Step 00052250 L_out=0.137 Acc=0.98 ---104982.69 ms/batch
Step 00052300 L_out=0.512 Acc=0.90 ---128443.63 ms/batch
Step 00052350 L_out=0.260 Acc=0.93 ---151572.21 ms/batch
Step 00052400 L_out=0.362 Acc=0.84 ---175196.41 ms/batch
Step 00052450 L_out=0.234 Acc=0.94 ---198560.58 ms/batch
Step 00052500 L_out=0.080 Acc=0.98 ---222124.98 ms/batch
Step 00052550 L_out=0.116 Acc=0.95 ---245719.42 ms/batch
Step 00052600 L_out=0.577 Acc=0.86 ---269139.48 ms/batch
Step 00052650 L_out=0.126 Acc=0.96 ---292655.18 ms/batch
Round 4 | epoch=18, train costTime=312.6815564632416, | total_activation_sum=26743247
Round 4 | epoch=19, current m_iou=0.7108786873520025
Round 4 | Best m_IoU is: 0.711, OA is: 0.928 | val costTime=118.06846189498901
Round 4 | ****EPOCH 19****
Step 00052700 L_out=0.095 Acc=0.96 --- 3235.13 ms/batch
Step 00052750 L_out=0.107 Acc=0.96 ---26696.77 ms/batch
Step 00052800 L_out=0.180 Acc=0.97 ---50030.08 ms/batch
Step 00052850 L_out=0.203 Acc=0.95 ---73229.53 ms/batch
Step 00052900 L_out=1.040 Acc=0.89 ---96830.50 ms/batch
Step 00052950 L_out=0.329 Acc=0.90 ---120099.57 ms/batch
Step 00053000 L_out=0.330 Acc=0.94 ---143630.13 ms/batch
Step 00053050 L_out=0.166 Acc=0.92 ---167162.35 ms/batch
Step 00053100 L_out=0.334 Acc=0.92 ---190551.18 ms/batch
Step 00053150 L_out=0.241 Acc=0.94 ---214082.50 ms/batch
Step 00053200 L_out=0.107 Acc=0.96 ---237492.06 ms/batch
Step 00053250 L_out=0.126 Acc=0.96 ---261184.84 ms/batch
Step 00053300 L_out=0.136 Acc=0.96 ---284584.20 ms/batch
Step 00053350 L_out=0.099 Acc=0.97 ---308028.81 ms/batch
Round 4 | epoch=19, train costTime=312.6779627799988, | total_activation_sum=26716193
Round 4 | epoch=20, current m_iou=0.7005163159467458
Round 4 | Best m_IoU is: 0.711, OA is: 0.928 | val costTime=117.47218585014343
Round 4 | ****EPOCH 20****
Step 00053400 L_out=0.196 Acc=0.93 ---18804.40 ms/batch
Step 00053450 L_out=0.320 Acc=0.93 ---42334.41 ms/batch
Step 00053500 L_out=0.176 Acc=0.95 ---65857.94 ms/batch
Step 00053550 L_out=0.151 Acc=0.95 ---89378.15 ms/batch
Step 00053600 L_out=0.109 Acc=0.97 ---112870.16 ms/batch
Step 00053650 L_out=0.047 Acc=0.97 ---136195.51 ms/batch
Step 00053700 L_out=0.168 Acc=0.96 ---159603.10 ms/batch
Step 00053750 L_out=0.228 Acc=0.92 ---183104.69 ms/batch
Step 00053800 L_out=0.158 Acc=0.95 ---206573.89 ms/batch
Step 00053850 L_out=0.473 Acc=0.86 ---230031.95 ms/batch
Step 00053900 L_out=0.114 Acc=0.95 ---253524.86 ms/batch
Step 00053950 L_out=0.198 Acc=0.94 ---276809.07 ms/batch
Step 00054000 L_out=0.143 Acc=0.95 ---300436.46 ms/batch
Round 4 | epoch=20, train costTime=313.30409955978394, | total_activation_sum=26797881
Round 4 | epoch=21, current m_iou=0.7089692240197023
Round 4 | Best m_IoU is: 0.711, OA is: 0.928 | val costTime=118.1752245426178
Round 4 | ****EPOCH 21****
Step 00054050 L_out=0.109 Acc=0.94 ---10518.59 ms/batch
Step 00054100 L_out=0.085 Acc=0.98 ---33885.30 ms/batch
Step 00054150 L_out=0.293 Acc=0.97 ---57222.35 ms/batch
Step 00054200 L_out=0.106 Acc=0.96 ---80683.99 ms/batch
Step 00054250 L_out=0.247 Acc=0.97 ---103917.78 ms/batch
Step 00054300 L_out=0.100 Acc=0.97 ---127139.67 ms/batch
Step 00054350 L_out=0.182 Acc=0.94 ---150360.40 ms/batch
Step 00054400 L_out=0.104 Acc=0.95 ---173810.22 ms/batch
Step 00054450 L_out=0.122 Acc=0.96 ---197207.91 ms/batch
Step 00054500 L_out=0.217 Acc=0.86 ---220526.43 ms/batch
Step 00054550 L_out=0.255 Acc=0.93 ---244113.46 ms/batch
Step 00054600 L_out=0.154 Acc=0.97 ---267766.50 ms/batch
Step 00054650 L_out=0.069 Acc=0.99 ---291075.37 ms/batch
Round 4 | epoch=21, train costTime=311.71970772743225, | total_activation_sum=26787295
Round 4 | epoch=22, current m_iou=0.7106304154115517
Round 4 | Best m_IoU is: 0.711, OA is: 0.928 | val costTime=118.67353415489197
Round 4 | ****EPOCH 22****
Step 00054700 L_out=0.335 Acc=0.91 --- 2765.89 ms/batch
Step 00054750 L_out=0.126 Acc=0.93 ---26138.84 ms/batch
Step 00054800 L_out=0.161 Acc=0.97 ---49606.63 ms/batch
Step 00054850 L_out=0.249 Acc=0.86 ---72905.51 ms/batch
Step 00054900 L_out=0.170 Acc=0.94 ---96295.08 ms/batch
Step 00054950 L_out=0.151 Acc=0.92 ---119693.62 ms/batch
Step 00055000 L_out=0.177 Acc=0.92 ---143242.19 ms/batch
Step 00055050 L_out=0.937 Acc=0.95 ---166690.08 ms/batch
Step 00055100 L_out=0.187 Acc=0.93 ---190345.82 ms/batch
Step 00055150 L_out=0.236 Acc=0.94 ---213889.08 ms/batch
Step 00055200 L_out=0.252 Acc=0.94 ---237402.23 ms/batch
Step 00055250 L_out=0.229 Acc=0.94 ---260794.45 ms/batch
Step 00055300 L_out=0.250 Acc=0.88 ---284033.60 ms/batch
Step 00055350 L_out=0.020 Acc=0.96 ---307537.93 ms/batch
Round 4 | epoch=22, train costTime=312.7555990219116, | total_activation_sum=26540954
Round 4 | epoch=23, current m_iou=0.7145720472907826
Round 4 | Best m_IoU is: 0.715, OA is: 0.929 | val costTime=118.5662682056427
Round 4 | ****EPOCH 23****
Step 00055400 L_out=0.125 Acc=0.95 ---18353.41 ms/batch
Step 00055450 L_out=0.261 Acc=0.97 ---41886.76 ms/batch
Step 00055500 L_out=0.345 Acc=0.91 ---65407.25 ms/batch
Step 00055550 L_out=0.156 Acc=0.97 ---89019.68 ms/batch
Step 00055600 L_out=0.156 Acc=0.96 ---112344.60 ms/batch
Step 00055650 L_out=0.147 Acc=0.92 ---135760.27 ms/batch
Step 00055700 L_out=0.083 Acc=0.97 ---159307.10 ms/batch
Step 00055750 L_out=0.120 Acc=0.96 ---182535.75 ms/batch
Step 00055800 L_out=0.785 Acc=0.91 ---205845.17 ms/batch
Step 00055850 L_out=0.200 Acc=0.96 ---229209.86 ms/batch
Step 00055900 L_out=0.164 Acc=0.95 ---252524.47 ms/batch
Step 00055950 L_out=0.216 Acc=0.93 ---275918.29 ms/batch
Step 00056000 L_out=0.143 Acc=0.95 ---299187.83 ms/batch
Round 4 | epoch=23, train costTime=312.38135862350464, | total_activation_sum=26591672
Round 4 | epoch=24, current m_iou=0.7185817279043965
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=118.14932322502136
Round 4 | ****EPOCH 24****
Step 00056050 L_out=0.102 Acc=0.96 ---10284.45 ms/batch
Step 00056100 L_out=0.102 Acc=0.96 ---33858.58 ms/batch
Step 00056150 L_out=0.230 Acc=0.97 ---57317.81 ms/batch
Step 00056200 L_out=0.185 Acc=0.89 ---80697.79 ms/batch
Step 00056250 L_out=0.249 Acc=0.88 ---104249.08 ms/batch
Step 00056300 L_out=0.231 Acc=0.92 ---127517.98 ms/batch
Step 00056350 L_out=0.512 Acc=0.90 ---150834.04 ms/batch
Step 00056400 L_out=0.166 Acc=0.97 ---174634.96 ms/batch
Step 00056450 L_out=0.070 Acc=0.94 ---197979.75 ms/batch
Step 00056500 L_out=0.206 Acc=0.93 ---221490.29 ms/batch
Step 00056550 L_out=0.231 Acc=0.93 ---244783.51 ms/batch
Step 00056600 L_out=0.138 Acc=0.96 ---268353.69 ms/batch
Step 00056650 L_out=0.183 Acc=0.95 ---291846.18 ms/batch
Round 4 | epoch=24, train costTime=312.93334341049194, | total_activation_sum=26695997
Round 4 | epoch=25, current m_iou=0.7155486527111583
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=117.61946678161621
Round 4 | ****EPOCH 25****
Step 00056700 L_out=0.182 Acc=0.94 --- 2363.76 ms/batch
Step 00056750 L_out=0.094 Acc=0.95 ---25634.12 ms/batch
Step 00056800 L_out=0.076 Acc=0.97 ---48925.87 ms/batch
Step 00056850 L_out=0.077 Acc=0.95 ---72310.74 ms/batch
Step 00056900 L_out=0.681 Acc=0.92 ---95648.88 ms/batch
Step 00056950 L_out=0.086 Acc=0.96 ---119115.48 ms/batch
Step 00057000 L_out=0.048 Acc=0.98 ---142587.19 ms/batch
Step 00057050 L_out=0.139 Acc=0.94 ---165985.91 ms/batch
Step 00057100 L_out=0.139 Acc=0.95 ---189258.11 ms/batch
Step 00057150 L_out=0.081 Acc=0.98 ---212704.36 ms/batch
Step 00057200 L_out=0.280 Acc=0.91 ---236160.05 ms/batch
Step 00057250 L_out=0.157 Acc=0.90 ---259369.33 ms/batch
Step 00057300 L_out=0.128 Acc=0.92 ---282877.07 ms/batch
Step 00057350 L_out=0.154 Acc=0.93 ---306136.30 ms/batch
Round 4 | epoch=25, train costTime=311.80216813087463, | total_activation_sum=26694037
Round 4 | epoch=26, current m_iou=0.7173975937720861
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=118.23475575447083
Round 4 | ****EPOCH 26****
Step 00057400 L_out=0.086 Acc=0.93 ---17837.11 ms/batch
Step 00057450 L_out=0.192 Acc=0.94 ---41301.34 ms/batch
Step 00057500 L_out=0.184 Acc=0.95 ---64411.36 ms/batch
Step 00057550 L_out=0.145 Acc=0.97 ---87819.53 ms/batch
Step 00057600 L_out=0.121 Acc=0.94 ---111390.08 ms/batch
Step 00057650 L_out=0.214 Acc=0.94 ---134753.75 ms/batch
Step 00057700 L_out=0.059 Acc=0.98 ---158098.27 ms/batch
Step 00057750 L_out=0.154 Acc=0.95 ---181608.71 ms/batch
Step 00057800 L_out=0.049 Acc=0.96 ---204924.41 ms/batch
Step 00057850 L_out=0.601 Acc=0.82 ---228067.84 ms/batch
Step 00057900 L_out=0.231 Acc=0.90 ---251415.47 ms/batch
Step 00057950 L_out=0.292 Acc=0.95 ---274727.18 ms/batch
Step 00058000 L_out=0.147 Acc=0.96 ---297996.63 ms/batch
Round 4 | epoch=26, train costTime=311.40596413612366, | total_activation_sum=26784778
Round 4 | epoch=27, current m_iou=0.7166769993076223
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=117.73099780082703
Round 4 | ****EPOCH 27****
Step 00058050 L_out=0.167 Acc=0.96 --- 9959.62 ms/batch
Step 00058100 L_out=1.498 Acc=0.87 ---33254.32 ms/batch
Step 00058150 L_out=0.239 Acc=0.93 ---56468.25 ms/batch
Step 00058200 L_out=0.419 Acc=0.87 ---80091.11 ms/batch
Step 00058250 L_out=0.291 Acc=0.92 ---103611.26 ms/batch
Step 00058300 L_out=0.161 Acc=0.95 ---126994.11 ms/batch
Step 00058350 L_out=0.215 Acc=0.94 ---150493.59 ms/batch
Step 00058400 L_out=0.161 Acc=0.95 ---173976.87 ms/batch
Step 00058450 L_out=0.244 Acc=0.93 ---197480.59 ms/batch
Step 00058500 L_out=0.146 Acc=0.94 ---220754.57 ms/batch
Step 00058550 L_out=0.363 Acc=0.93 ---244075.68 ms/batch
Step 00058600 L_out=0.157 Acc=0.94 ---267444.56 ms/batch
Step 00058650 L_out=0.496 Acc=0.90 ---291082.19 ms/batch
Round 4 | epoch=27, train costTime=312.6139361858368, | total_activation_sum=26742336
Round 4 | epoch=28, current m_iou=0.7169488876098767
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=117.73427271842957
Round 4 | ****EPOCH 28****
Step 00058700 L_out=0.133 Acc=0.96 --- 1925.24 ms/batch
Step 00058750 L_out=0.084 Acc=0.96 ---25184.85 ms/batch
Step 00058800 L_out=0.132 Acc=0.90 ---48687.21 ms/batch
Step 00058850 L_out=0.084 Acc=0.96 ---72245.09 ms/batch
Step 00058900 L_out=0.185 Acc=0.95 ---95672.37 ms/batch
Step 00058950 L_out=0.082 Acc=0.94 ---119036.18 ms/batch
Step 00059000 L_out=0.062 Acc=0.98 ---142448.76 ms/batch
Step 00059050 L_out=0.137 Acc=0.95 ---165835.56 ms/batch
Step 00059100 L_out=0.088 Acc=0.98 ---189255.83 ms/batch
Step 00059150 L_out=0.195 Acc=0.93 ---212712.70 ms/batch
Step 00059200 L_out=0.163 Acc=0.95 ---236273.29 ms/batch
Step 00059250 L_out=0.089 Acc=0.95 ---259683.97 ms/batch
Step 00059300 L_out=0.156 Acc=0.95 ---283068.50 ms/batch
Step 00059350 L_out=0.113 Acc=0.96 ---306284.79 ms/batch
Round 4 | epoch=28, train costTime=312.3047184944153, | total_activation_sum=26617968
Round 4 | epoch=29, current m_iou=0.7154223350238362
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=117.85274076461792
Round 4 | ****EPOCH 29****
Step 00059400 L_out=0.167 Acc=0.95 ---17382.03 ms/batch
Step 00059450 L_out=0.226 Acc=0.93 ---40969.57 ms/batch
Step 00059500 L_out=0.055 Acc=0.97 ---64215.28 ms/batch
Step 00059550 L_out=0.361 Acc=0.92 ---87497.91 ms/batch
Step 00059600 L_out=0.086 Acc=0.97 ---110755.30 ms/batch
Step 00059650 L_out=0.112 Acc=0.96 ---134211.96 ms/batch
Step 00059700 L_out=0.117 Acc=0.96 ---157649.17 ms/batch
Step 00059750 L_out=0.149 Acc=0.94 ---181034.03 ms/batch
Step 00059800 L_out=0.070 Acc=0.94 ---204355.52 ms/batch
Step 00059850 L_out=0.146 Acc=0.97 ---227802.06 ms/batch
Step 00059900 L_out=0.086 Acc=0.95 ---251383.80 ms/batch
Step 00059950 L_out=0.199 Acc=0.95 ---274751.45 ms/batch
Step 00060000 L_out=0.112 Acc=0.96 ---298265.35 ms/batch
Round 4 | epoch=29, train costTime=312.219450712204, | total_activation_sum=26678694
Round 4 | epoch=30, current m_iou=0.7132631769097
Round 4 | Best m_IoU is: 0.719, OA is: 0.932 | val costTime=117.47840166091919
Round 4 | ****EPOCH 30****
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.542 Acc=0.06 ---25484.41 ms/batch
Step 00000100 L_out=1.267 Acc=0.45 ---48996.96 ms/batch
Step 00000150 L_out=1.374 Acc=0.43 ---72558.67 ms/batch
Step 00000200 L_out=0.724 Acc=0.21 ---96188.10 ms/batch
Step 00000250 L_out=0.272 Acc=0.46 ---119992.66 ms/batch
Step 00000300 L_out=0.855 Acc=0.39 ---143608.48 ms/batch
Step 00000350 L_out=0.930 Acc=0.41 ---167701.63 ms/batch
Step 00000400 L_out=0.739 Acc=0.34 ---191511.22 ms/batch
Step 00000450 L_out=0.444 Acc=0.21 ---215109.21 ms/batch
Step 00000500 L_out=0.566 Acc=0.38 ---238704.99 ms/batch
Step 00000550 L_out=0.322 Acc=0.55 ---262425.89 ms/batch
Step 00000600 L_out=0.774 Acc=0.47 ---286103.13 ms/batch
Step 00000650 L_out=0.734 Acc=0.57 ---309830.26 ms/batch
Round 2 | epoch=0, train costTime=317.7939052581787, | total_activation_sum=13593177
Round 2 | ****EPOCH 1****
Step 00000700 L_out=1.431 Acc=0.70 ---15661.19 ms/batch
Step 00000750 L_out=0.712 Acc=0.61 ---39021.65 ms/batch
Step 00000800 L_out=1.332 Acc=0.46 ---62613.37 ms/batch
Step 00000850 L_out=0.863 Acc=0.31 ---86244.94 ms/batch
Step 00000900 L_out=0.634 Acc=0.25 ---109656.33 ms/batch
Step 00000950 L_out=0.357 Acc=0.76 ---133474.58 ms/batch
Step 00001000 L_out=0.119 Acc=0.58 ---157153.76 ms/batch
Step 00001050 L_out=0.513 Acc=0.46 ---180879.72 ms/batch
Step 00001100 L_out=0.691 Acc=0.53 ---204246.27 ms/batch
Step 00001150 L_out=0.362 Acc=0.71 ---227783.92 ms/batch
Step 00001200 L_out=0.466 Acc=0.43 ---251235.26 ms/batch
Step 00001250 L_out=0.165 Acc=0.57 ---274754.41 ms/batch
Step 00001300 L_out=0.844 Acc=0.40 ---298331.35 ms/batch
Round 2 | epoch=1, train costTime=314.23958706855774, | total_activation_sum=11971640
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.977 Acc=0.61 --- 7601.54 ms/batch
Step 00001400 L_out=0.490 Acc=0.75 ---31170.49 ms/batch
Step 00001450 L_out=1.926 Acc=0.24 ---54707.11 ms/batch
Step 00001500 L_out=0.536 Acc=0.58 ---78207.72 ms/batch
Step 00001550 L_out=0.409 Acc=0.66 ---101823.54 ms/batch
Step 00001600 L_out=0.569 Acc=0.60 ---125215.36 ms/batch
Step 00001650 L_out=0.472 Acc=0.81 ---148648.87 ms/batch
Step 00001700 L_out=0.439 Acc=0.66 ---172484.63 ms/batch
Step 00001750 L_out=0.321 Acc=0.42 ---196316.23 ms/batch
Step 00001800 L_out=1.952 Acc=0.55 ---219893.15 ms/batch
Step 00001850 L_out=0.214 Acc=0.67 ---243246.40 ms/batch
Step 00001900 L_out=0.327 Acc=0.65 ---266985.88 ms/batch
Step 00001950 L_out=0.118 Acc=0.75 ---290605.52 ms/batch
Step 00002000 L_out=0.511 Acc=0.40 ---314118.12 ms/batch
Round 2 | epoch=2, train costTime=314.5808744430542, | total_activation_sum=11876170
Round 2 | ****EPOCH 3****
Step 00002050 L_out=0.633 Acc=0.51 ---23000.52 ms/batch
Step 00002100 L_out=0.820 Acc=0.61 ---46444.52 ms/batch
Step 00002150 L_out=0.189 Acc=0.57 ---69970.06 ms/batch
Step 00002200 L_out=0.289 Acc=0.72 ---93626.04 ms/batch
Step 00002250 L_out=0.387 Acc=0.73 ---117199.39 ms/batch
Step 00002300 L_out=0.382 Acc=0.56 ---140624.30 ms/batch
Step 00002350 L_out=0.602 Acc=0.68 ---164039.56 ms/batch
Step 00002400 L_out=0.825 Acc=0.51 ---187719.15 ms/batch
Step 00002450 L_out=0.253 Acc=0.45 ---211288.74 ms/batch
Step 00002500 L_out=0.508 Acc=0.62 ---235063.30 ms/batch
Step 00002550 L_out=0.685 Acc=0.58 ---258686.51 ms/batch
Step 00002600 L_out=0.403 Acc=0.60 ---282260.28 ms/batch
Step 00002650 L_out=0.506 Acc=0.57 ---305945.95 ms/batch
Round 2 | epoch=3, train costTime=314.392879486084, | total_activation_sum=11967832
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.415 Acc=0.75 ---15033.09 ms/batch
Step 00002750 L_out=0.512 Acc=0.71 ---38630.89 ms/batch
Step 00002800 L_out=0.462 Acc=0.52 ---62416.63 ms/batch
Step 00002850 L_out=0.772 Acc=0.63 ---85875.53 ms/batch
Step 00002900 L_out=0.573 Acc=0.63 ---109405.02 ms/batch
Step 00002950 L_out=0.570 Acc=0.81 ---132838.60 ms/batch
Step 00003000 L_out=0.391 Acc=0.60 ---156567.86 ms/batch
Step 00003050 L_out=0.296 Acc=0.61 ---180219.38 ms/batch
Step 00003100 L_out=0.770 Acc=0.80 ---203906.95 ms/batch
Step 00003150 L_out=0.839 Acc=0.59 ---227291.41 ms/batch
Step 00003200 L_out=0.358 Acc=0.87 ---251039.38 ms/batch
Step 00003250 L_out=0.868 Acc=0.65 ---274640.67 ms/batch
Step 00003300 L_out=0.291 Acc=0.63 ---298226.63 ms/batch
Round 2 | epoch=4, train costTime=314.69440841674805, | total_activation_sum=11798669
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.457 Acc=0.76 --- 6990.79 ms/batch
Step 00003400 L_out=0.250 Acc=0.73 ---30475.95 ms/batch
Step 00003450 L_out=0.327 Acc=0.76 ---53917.65 ms/batch
Step 00003500 L_out=0.189 Acc=0.73 ---77527.34 ms/batch
Step 00003550 L_out=0.550 Acc=0.65 ---101027.12 ms/batch
Step 00003600 L_out=0.626 Acc=0.51 ---124747.65 ms/batch
Step 00003650 L_out=0.954 Acc=0.47 ---148312.92 ms/batch
Step 00003700 L_out=0.733 Acc=0.37 ---171695.12 ms/batch
Step 00003750 L_out=0.735 Acc=0.62 ---195212.33 ms/batch
Step 00003800 L_out=0.683 Acc=0.68 ---218873.64 ms/batch
Step 00003850 L_out=0.854 Acc=0.52 ---242265.70 ms/batch
Step 00003900 L_out=0.146 Acc=0.89 ---265821.56 ms/batch
Step 00003950 L_out=0.492 Acc=0.80 ---289570.88 ms/batch
Step 00004000 L_out=0.336 Acc=0.57 ---313197.72 ms/batch
Round 2 | epoch=5, train costTime=314.1522262096405, | total_activation_sum=12049164
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.598 Acc=0.84 ---22395.97 ms/batch
Step 00004100 L_out=0.228 Acc=0.83 ---45849.26 ms/batch
Step 00004150 L_out=0.415 Acc=0.72 ---69343.49 ms/batch
Step 00004200 L_out=0.354 Acc=0.74 ---92934.04 ms/batch
Step 00004250 L_out=0.250 Acc=0.74 ---116500.15 ms/batch
Step 00004300 L_out=0.931 Acc=0.42 ---139969.18 ms/batch
Step 00004350 L_out=0.229 Acc=0.79 ---163185.58 ms/batch
Step 00004400 L_out=0.596 Acc=0.53 ---186506.93 ms/batch
Step 00004450 L_out=0.371 Acc=0.82 ---210069.98 ms/batch
Step 00004500 L_out=0.319 Acc=0.80 ---233695.11 ms/batch
Step 00004550 L_out=0.619 Acc=0.65 ---257232.32 ms/batch
Step 00004600 L_out=0.392 Acc=0.77 ---280632.00 ms/batch
Step 00004650 L_out=0.447 Acc=0.71 ---304357.02 ms/batch
Round 2 | epoch=6, train costTime=313.1342282295227, | total_activation_sum=12029342
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.615 Acc=0.81 ---14790.34 ms/batch
Step 00004750 L_out=0.236 Acc=0.76 ---38260.71 ms/batch
Step 00004800 L_out=0.308 Acc=0.74 ---61912.10 ms/batch
Step 00004850 L_out=0.306 Acc=0.77 ---85333.31 ms/batch
Step 00004900 L_out=0.472 Acc=0.68 ---108815.62 ms/batch
Step 00004950 L_out=0.309 Acc=0.86 ---132276.39 ms/batch
Step 00005000 L_out=0.181 Acc=0.82 ---155630.29 ms/batch
Step 00005050 L_out=0.138 Acc=0.70 ---179155.53 ms/batch
Step 00005100 L_out=0.111 Acc=0.80 ---202753.63 ms/batch
Step 00005150 L_out=0.097 Acc=0.75 ---226291.05 ms/batch
Step 00005200 L_out=0.605 Acc=0.55 ---249711.35 ms/batch
Step 00005250 L_out=0.852 Acc=0.54 ---273218.56 ms/batch
Step 00005300 L_out=0.718 Acc=0.78 ---296619.84 ms/batch
Round 2 | epoch=7, train costTime=313.47694635391235, | total_activation_sum=12015640
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.861 Acc=0.73 --- 6605.91 ms/batch
Step 00005400 L_out=0.211 Acc=0.67 ---29954.78 ms/batch
Step 00005450 L_out=0.184 Acc=0.82 ---53543.47 ms/batch
Step 00005500 L_out=0.316 Acc=0.81 ---77301.10 ms/batch
Step 00005550 L_out=0.164 Acc=0.87 ---100827.39 ms/batch
Step 00005600 L_out=0.266 Acc=0.77 ---124455.17 ms/batch
Step 00005650 L_out=0.159 Acc=0.85 ---147884.07 ms/batch
Step 00005700 L_out=0.226 Acc=0.60 ---171286.45 ms/batch
Step 00005750 L_out=0.387 Acc=0.69 ---194778.89 ms/batch
Step 00005800 L_out=0.337 Acc=0.88 ---218341.66 ms/batch
Step 00005850 L_out=0.831 Acc=0.61 ---241760.44 ms/batch
Step 00005900 L_out=0.301 Acc=0.63 ---265298.84 ms/batch
Step 00005950 L_out=0.453 Acc=0.76 ---288691.93 ms/batch
Step 00006000 L_out=0.415 Acc=0.70 ---312204.98 ms/batch
Round 2 | epoch=8, train costTime=313.6369159221649, | total_activation_sum=11968020
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.261 Acc=0.90 ---22105.10 ms/batch
Step 00006100 L_out=0.257 Acc=0.77 ---45604.05 ms/batch
Step 00006150 L_out=0.381 Acc=0.78 ---69162.37 ms/batch
Step 00006200 L_out=0.322 Acc=0.87 ---92831.99 ms/batch
Step 00006250 L_out=0.334 Acc=0.67 ---116451.77 ms/batch
Step 00006300 L_out=0.349 Acc=0.66 ---140210.88 ms/batch
Step 00006350 L_out=0.105 Acc=0.74 ---163773.12 ms/batch
Step 00006400 L_out=0.102 Acc=0.91 ---187473.51 ms/batch
Step 00006450 L_out=0.104 Acc=0.85 ---211180.53 ms/batch
Step 00006500 L_out=0.377 Acc=0.74 ---234870.31 ms/batch
Step 00006550 L_out=0.470 Acc=0.68 ---258486.06 ms/batch
Step 00006600 L_out=0.795 Acc=0.74 ---282106.85 ms/batch
Step 00006650 L_out=0.167 Acc=0.74 ---305575.92 ms/batch
Round 2 | epoch=9, train costTime=315.01181650161743, | total_activation_sum=11958643
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.597 Acc=0.69 ---14064.35 ms/batch
Step 00006750 L_out=0.536 Acc=0.90 ---37488.89 ms/batch
Step 00006800 L_out=0.146 Acc=0.80 ---60993.40 ms/batch
Step 00006850 L_out=0.440 Acc=0.74 ---84595.62 ms/batch
Step 00006900 L_out=0.160 Acc=0.74 ---108037.83 ms/batch
Step 00006950 L_out=0.402 Acc=0.79 ---131732.40 ms/batch
Step 00007000 L_out=0.078 Acc=0.88 ---155001.56 ms/batch
Step 00007050 L_out=0.197 Acc=0.88 ---178494.46 ms/batch
Step 00007100 L_out=0.202 Acc=0.82 ---202001.65 ms/batch
Step 00007150 L_out=0.431 Acc=0.65 ---225349.28 ms/batch
Step 00007200 L_out=0.497 Acc=0.75 ---249060.29 ms/batch
Step 00007250 L_out=0.175 Acc=0.84 ---272458.82 ms/batch
Step 00007300 L_out=0.324 Acc=0.78 ---296174.25 ms/batch
Round 2 | epoch=10, train costTime=313.6565263271332, | total_activation_sum=11998239
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.168 Acc=0.76 --- 6020.09 ms/batch
Step 00007400 L_out=0.164 Acc=0.85 ---29538.50 ms/batch
Step 00007450 L_out=0.284 Acc=0.85 ---53161.83 ms/batch
Step 00007500 L_out=0.427 Acc=0.72 ---76408.92 ms/batch
Step 00007550 L_out=0.181 Acc=0.70 ---99978.46 ms/batch
Step 00007600 L_out=0.411 Acc=0.70 ---123417.57 ms/batch
Step 00007650 L_out=0.629 Acc=0.66 ---147151.26 ms/batch
Step 00007700 L_out=0.307 Acc=0.71 ---170813.04 ms/batch
Step 00007750 L_out=0.106 Acc=0.84 ---194172.22 ms/batch
Step 00007800 L_out=0.207 Acc=0.80 ---217447.33 ms/batch
Step 00007850 L_out=0.321 Acc=0.82 ---241205.35 ms/batch
Step 00007900 L_out=0.211 Acc=0.79 ---264874.28 ms/batch
Step 00007950 L_out=0.314 Acc=0.83 ---288478.24 ms/batch
Step 00008000 L_out=0.322 Acc=0.79 ---312031.53 ms/batch
Round 2 | epoch=11, train costTime=313.8992066383362, | total_activation_sum=11979310
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.434 Acc=0.82 ---21566.28 ms/batch
Step 00008100 L_out=0.395 Acc=0.74 ---45131.54 ms/batch
Step 00008150 L_out=0.211 Acc=0.78 ---68553.64 ms/batch
Step 00008200 L_out=0.160 Acc=0.86 ---92021.64 ms/batch
Step 00008250 L_out=0.166 Acc=0.76 ---115437.78 ms/batch
Step 00008300 L_out=0.338 Acc=0.66 ---139101.03 ms/batch
Step 00008350 L_out=0.170 Acc=0.77 ---162719.60 ms/batch
Step 00008400 L_out=0.638 Acc=0.86 ---186220.04 ms/batch
Step 00008450 L_out=0.264 Acc=0.83 ---209663.69 ms/batch
Step 00008500 L_out=0.264 Acc=0.79 ---233185.95 ms/batch
Step 00008550 L_out=0.103 Acc=0.79 ---256452.95 ms/batch
Step 00008600 L_out=0.260 Acc=0.73 ---279938.75 ms/batch
Step 00008650 L_out=0.169 Acc=0.90 ---303290.33 ms/batch
Round 2 | epoch=12, train costTime=313.10137033462524, | total_activation_sum=11992338
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.195 Acc=0.77 ---13504.86 ms/batch
Step 00008750 L_out=0.093 Acc=0.79 ---37283.09 ms/batch
Step 00008800 L_out=0.227 Acc=0.86 ---60724.56 ms/batch
Step 00008850 L_out=0.737 Acc=0.82 ---84302.27 ms/batch
Step 00008900 L_out=0.362 Acc=0.75 ---107911.51 ms/batch
Step 00008950 L_out=0.208 Acc=0.76 ---131552.97 ms/batch
Step 00009000 L_out=0.088 Acc=0.83 ---155146.87 ms/batch
Step 00009050 L_out=0.098 Acc=0.81 ---178810.49 ms/batch
Step 00009100 L_out=0.029 Acc=0.80 ---202408.61 ms/batch
Step 00009150 L_out=0.226 Acc=0.83 ---225741.12 ms/batch
Step 00009200 L_out=0.221 Acc=0.89 ---249156.89 ms/batch
Step 00009250 L_out=0.267 Acc=0.76 ---272715.98 ms/batch
Step 00009300 L_out=0.193 Acc=0.86 ---296308.00 ms/batch
Round 2 | epoch=13, train costTime=314.1838312149048, | total_activation_sum=11943176
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.332 Acc=0.72 --- 5541.17 ms/batch
Step 00009400 L_out=0.310 Acc=0.78 ---28960.22 ms/batch
Step 00009450 L_out=0.178 Acc=0.85 ---52331.64 ms/batch
Step 00009500 L_out=0.238 Acc=0.83 ---75678.09 ms/batch
Step 00009550 L_out=0.094 Acc=0.83 ---99105.01 ms/batch
Step 00009600 L_out=0.292 Acc=0.79 ---122874.59 ms/batch
Step 00009650 L_out=0.075 Acc=0.95 ---146442.56 ms/batch
Step 00009700 L_out=0.444 Acc=0.78 ---169901.04 ms/batch
Step 00009750 L_out=0.119 Acc=0.88 ---193519.40 ms/batch
Step 00009800 L_out=0.145 Acc=0.87 ---217223.49 ms/batch
Step 00009850 L_out=0.156 Acc=0.84 ---240837.11 ms/batch
Step 00009900 L_out=0.099 Acc=0.91 ---264217.97 ms/batch
Step 00009950 L_out=0.217 Acc=0.82 ---287692.94 ms/batch
Step 00010000 L_out=0.251 Acc=0.77 ---311306.71 ms/batch
Round 2 | epoch=14, train costTime=313.72131156921387, | total_activation_sum=12016406
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.497 Acc=0.85 ---21365.70 ms/batch
Step 00010100 L_out=0.197 Acc=0.91 ---44956.43 ms/batch
Step 00010150 L_out=0.113 Acc=0.89 ---68339.20 ms/batch
Step 00010200 L_out=0.216 Acc=0.77 ---91947.55 ms/batch
Step 00010250 L_out=0.198 Acc=0.69 ---115492.38 ms/batch
Step 00010300 L_out=0.113 Acc=0.82 ---139034.11 ms/batch
Step 00010350 L_out=0.125 Acc=0.85 ---162607.79 ms/batch
Step 00010400 L_out=0.426 Acc=0.76 ---186156.46 ms/batch
Step 00010450 L_out=0.111 Acc=0.90 ---209708.91 ms/batch
Step 00010500 L_out=0.304 Acc=0.85 ---233100.71 ms/batch
Step 00010550 L_out=0.069 Acc=0.89 ---256675.35 ms/batch
Step 00010600 L_out=0.162 Acc=0.81 ---280564.58 ms/batch
Step 00010650 L_out=0.220 Acc=0.90 ---304068.59 ms/batch
Round 2 | epoch=15, train costTime=314.40541791915894, | total_activation_sum=12178116
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.053 Acc=0.92 ---13398.87 ms/batch
Step 00010750 L_out=0.229 Acc=0.88 ---37097.42 ms/batch
Step 00010800 L_out=0.247 Acc=0.81 ---60666.64 ms/batch
Step 00010850 L_out=0.094 Acc=0.79 ---84184.97 ms/batch
Step 00010900 L_out=0.170 Acc=0.87 ---107909.71 ms/batch
Step 00010950 L_out=0.303 Acc=0.83 ---131565.11 ms/batch
Step 00011000 L_out=0.384 Acc=0.87 ---154837.91 ms/batch
Step 00011050 L_out=0.544 Acc=0.92 ---178382.74 ms/batch
Step 00011100 L_out=0.104 Acc=0.87 ---202148.44 ms/batch
Step 00011150 L_out=0.118 Acc=0.88 ---225820.03 ms/batch
Step 00011200 L_out=0.142 Acc=0.86 ---249515.34 ms/batch
Step 00011250 L_out=0.241 Acc=0.84 ---273142.74 ms/batch
Step 00011300 L_out=0.165 Acc=0.83 ---296600.40 ms/batch
Round 2 | epoch=16, train costTime=314.87317991256714, | total_activation_sum=11888234
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.192 Acc=0.84 --- 5097.77 ms/batch
Step 00011400 L_out=0.605 Acc=0.71 ---28590.69 ms/batch
Step 00011450 L_out=0.288 Acc=0.75 ---52211.21 ms/batch
Step 00011500 L_out=0.159 Acc=0.88 ---75716.30 ms/batch
Step 00011550 L_out=0.154 Acc=0.76 ---99236.58 ms/batch
Step 00011600 L_out=0.273 Acc=0.77 ---122614.84 ms/batch
Step 00011650 L_out=0.198 Acc=0.88 ---146033.71 ms/batch
Step 00011700 L_out=1.662 Acc=0.71 ---169486.26 ms/batch
Step 00011750 L_out=0.079 Acc=0.87 ---192868.63 ms/batch
Step 00011800 L_out=0.162 Acc=0.81 ---216455.10 ms/batch
Step 00011850 L_out=0.076 Acc=0.89 ---239981.61 ms/batch
Step 00011900 L_out=0.161 Acc=0.84 ---263338.31 ms/batch
Step 00011950 L_out=0.109 Acc=0.86 ---286932.41 ms/batch
Step 00012000 L_out=0.102 Acc=0.89 ---310442.66 ms/batch
Round 2 | epoch=17, train costTime=313.3576135635376, | total_activation_sum=12094839
Round 2 | epoch=18, current m_iou=0.4595221571845647
Round 2 | Best m_IoU is: 0.460, OA is: 0.795 | val costTime=117.71960020065308
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.218 Acc=0.88 ---20684.24 ms/batch
Step 00012100 L_out=0.352 Acc=0.73 ---44222.67 ms/batch
Step 00012150 L_out=0.399 Acc=0.81 ---67632.21 ms/batch
Step 00012200 L_out=0.210 Acc=0.85 ---91006.99 ms/batch
Step 00012250 L_out=0.134 Acc=0.82 ---114288.18 ms/batch
Step 00012300 L_out=0.225 Acc=0.78 ---137795.23 ms/batch
Step 00012350 L_out=0.251 Acc=0.72 ---161219.88 ms/batch
Step 00012400 L_out=0.214 Acc=0.90 ---184878.27 ms/batch
Step 00012450 L_out=0.111 Acc=0.82 ---208403.11 ms/batch
Step 00012500 L_out=0.199 Acc=0.79 ---231912.26 ms/batch
Step 00012550 L_out=0.428 Acc=0.80 ---255357.19 ms/batch
Step 00012600 L_out=0.031 Acc=0.84 ---279001.39 ms/batch
Step 00012650 L_out=0.050 Acc=0.86 ---302482.59 ms/batch
Round 2 | epoch=18, train costTime=313.3868944644928, | total_activation_sum=11905379
Round 2 | epoch=19, current m_iou=0.5481702683734947
Round 2 | Best m_IoU is: 0.548, OA is: 0.863 | val costTime=117.3465473651886
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.430 Acc=0.87 ---12726.21 ms/batch
Step 00012750 L_out=0.063 Acc=0.85 ---36213.91 ms/batch
Step 00012800 L_out=0.175 Acc=0.91 ---59584.07 ms/batch
Step 00012850 L_out=0.133 Acc=0.92 ---83195.49 ms/batch
Step 00012900 L_out=0.083 Acc=0.62 ---106562.33 ms/batch
Step 00012950 L_out=0.088 Acc=0.89 ---130094.25 ms/batch
Step 00013000 L_out=0.290 Acc=0.79 ---153687.51 ms/batch
Step 00013050 L_out=0.023 Acc=0.87 ---177214.84 ms/batch
Step 00013100 L_out=0.139 Acc=0.86 ---200583.81 ms/batch
Step 00013150 L_out=0.308 Acc=0.67 ---224052.96 ms/batch
Step 00013200 L_out=0.193 Acc=0.85 ---247696.24 ms/batch
Step 00013250 L_out=0.090 Acc=0.87 ---271324.05 ms/batch
Step 00013300 L_out=0.025 Acc=0.87 ---294637.18 ms/batch
Round 2 | epoch=19, train costTime=313.41396284103394, | total_activation_sum=11906787
Round 2 | epoch=20, current m_iou=0.5594156718228684
Round 2 | Best m_IoU is: 0.559, OA is: 0.869 | val costTime=117.54135394096375
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.083 Acc=0.90 --- 4664.63 ms/batch
Step 00013400 L_out=0.249 Acc=0.85 ---28013.21 ms/batch
Step 00013450 L_out=0.227 Acc=0.78 ---51515.20 ms/batch
Step 00013500 L_out=0.118 Acc=0.92 ---75054.80 ms/batch
Step 00013550 L_out=0.112 Acc=0.87 ---98449.26 ms/batch
Step 00013600 L_out=0.126 Acc=0.81 ---121940.12 ms/batch
Step 00013650 L_out=0.280 Acc=0.91 ---145531.13 ms/batch
Step 00013700 L_out=0.292 Acc=0.78 ---168874.54 ms/batch
Step 00013750 L_out=0.415 Acc=0.72 ---192245.51 ms/batch
Step 00013800 L_out=0.080 Acc=0.89 ---215968.88 ms/batch
Step 00013850 L_out=0.061 Acc=0.91 ---239362.33 ms/batch
Step 00013900 L_out=0.275 Acc=0.61 ---262736.85 ms/batch
Step 00013950 L_out=0.075 Acc=0.86 ---286256.04 ms/batch
Step 00014000 L_out=0.222 Acc=0.80 ---309596.97 ms/batch
Round 2 | epoch=20, train costTime=312.85343503952026, | total_activation_sum=11914180
Round 2 | epoch=21, current m_iou=0.5634775082920247
Round 2 | Best m_IoU is: 0.563, OA is: 0.864 | val costTime=117.26055240631104
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.120 Acc=0.81 ---20352.91 ms/batch
Step 00014100 L_out=0.156 Acc=0.83 ---43879.76 ms/batch
Step 00014150 L_out=0.053 Acc=0.90 ---67235.12 ms/batch
Step 00014200 L_out=0.048 Acc=0.83 ---90729.41 ms/batch
Step 00014250 L_out=0.247 Acc=0.87 ---114059.82 ms/batch
Step 00014300 L_out=0.210 Acc=0.85 ---137886.23 ms/batch
Step 00014350 L_out=0.123 Acc=0.89 ---161565.08 ms/batch
Step 00014400 L_out=0.068 Acc=0.52 ---184986.23 ms/batch
Step 00014450 L_out=0.153 Acc=0.92 ---208578.86 ms/batch
Step 00014500 L_out=0.124 Acc=0.76 ---232171.35 ms/batch
Step 00014550 L_out=0.074 Acc=0.80 ---255636.71 ms/batch
Step 00014600 L_out=0.590 Acc=0.74 ---278960.92 ms/batch
Step 00014650 L_out=0.137 Acc=0.93 ---302224.02 ms/batch
Round 2 | epoch=21, train costTime=313.46406173706055, | total_activation_sum=12196967
Round 2 | epoch=22, current m_iou=0.55390389070635
Round 2 | Best m_IoU is: 0.563, OA is: 0.864 | val costTime=116.34582805633545
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.032 Acc=0.81 ---12341.76 ms/batch
Step 00014750 L_out=0.165 Acc=0.87 ---35746.89 ms/batch
Step 00014800 L_out=0.058 Acc=0.82 ---59167.46 ms/batch
Step 00014850 L_out=0.224 Acc=0.86 ---82508.43 ms/batch
Step 00014900 L_out=0.047 Acc=0.92 ---105884.87 ms/batch
Step 00014950 L_out=0.109 Acc=0.79 ---129320.45 ms/batch
Step 00015000 L_out=0.034 Acc=0.83 ---152818.69 ms/batch
Step 00015050 L_out=0.092 Acc=0.84 ---176421.04 ms/batch
Step 00015100 L_out=0.263 Acc=0.88 ---199697.67 ms/batch
Step 00015150 L_out=0.362 Acc=0.77 ---223260.13 ms/batch
Step 00015200 L_out=0.100 Acc=0.86 ---246671.37 ms/batch
Step 00015250 L_out=0.111 Acc=0.90 ---270322.00 ms/batch
Step 00015300 L_out=0.185 Acc=0.76 ---293808.64 ms/batch
Round 2 | epoch=22, train costTime=313.2333436012268, | total_activation_sum=11877607
Round 2 | epoch=23, current m_iou=0.5228361000157744
Round 2 | Best m_IoU is: 0.563, OA is: 0.864 | val costTime=116.88027930259705
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.042 Acc=0.92 --- 4156.11 ms/batch
Step 00015400 L_out=0.112 Acc=0.83 ---27558.33 ms/batch
Step 00015450 L_out=0.101 Acc=0.90 ---51238.45 ms/batch
Step 00015500 L_out=0.026 Acc=0.97 ---74764.63 ms/batch
Step 00015550 L_out=0.033 Acc=0.72 ---98176.03 ms/batch
Step 00015600 L_out=0.076 Acc=0.88 ---121621.86 ms/batch
Step 00015650 L_out=0.093 Acc=0.78 ---145092.48 ms/batch
Step 00015700 L_out=0.105 Acc=0.90 ---168731.90 ms/batch
Step 00015750 L_out=0.183 Acc=0.82 ---192111.61 ms/batch
Step 00015800 L_out=0.312 Acc=0.78 ---215726.01 ms/batch
Step 00015850 L_out=0.308 Acc=0.82 ---238932.34 ms/batch
Step 00015900 L_out=0.377 Acc=0.86 ---262297.35 ms/batch
Step 00015950 L_out=0.201 Acc=0.87 ---285729.48 ms/batch
Step 00016000 L_out=0.298 Acc=0.80 ---309309.04 ms/batch
Round 2 | epoch=23, train costTime=313.0253551006317, | total_activation_sum=11981247
Round 2 | epoch=24, current m_iou=0.5543996757171386
Round 2 | Best m_IoU is: 0.563, OA is: 0.864 | val costTime=116.62136149406433
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.353 Acc=0.78 ---19660.39 ms/batch
Step 00016100 L_out=0.152 Acc=0.88 ---43262.05 ms/batch
Step 00016150 L_out=0.041 Acc=0.91 ---66978.07 ms/batch
Step 00016200 L_out=0.224 Acc=0.90 ---90493.79 ms/batch
Step 00016250 L_out=0.109 Acc=0.92 ---113919.27 ms/batch
Step 00016300 L_out=0.195 Acc=0.78 ---137627.80 ms/batch
Step 00016350 L_out=0.060 Acc=0.93 ---161073.34 ms/batch
Step 00016400 L_out=0.419 Acc=0.76 ---184592.29 ms/batch
Step 00016450 L_out=0.101 Acc=0.93 ---207937.59 ms/batch
Step 00016500 L_out=0.021 Acc=0.90 ---231135.64 ms/batch
Step 00016550 L_out=0.266 Acc=0.82 ---254753.38 ms/batch
Step 00016600 L_out=0.163 Acc=0.86 ---278102.87 ms/batch
Step 00016650 L_out=0.196 Acc=0.91 ---301559.60 ms/batch
Round 2 | epoch=24, train costTime=313.33591389656067, | total_activation_sum=11722470
Round 2 | epoch=25, current m_iou=0.5749747540791883
Round 2 | Best m_IoU is: 0.575, OA is: 0.881 | val costTime=116.97704601287842
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.045 Acc=0.89 ---11736.79 ms/batch
Step 00016750 L_out=0.151 Acc=0.86 ---35024.64 ms/batch
Step 00016800 L_out=0.055 Acc=0.88 ---58507.36 ms/batch
Step 00016850 L_out=0.177 Acc=0.90 ---82193.32 ms/batch
Step 00016900 L_out=0.068 Acc=0.90 ---105807.89 ms/batch
Step 00016950 L_out=0.314 Acc=0.78 ---129602.70 ms/batch
Step 00017000 L_out=0.088 Acc=0.91 ---153215.15 ms/batch
Step 00017050 L_out=0.076 Acc=0.94 ---176596.74 ms/batch
Step 00017100 L_out=0.142 Acc=0.81 ---200130.83 ms/batch
Step 00017150 L_out=0.153 Acc=0.91 ---223659.20 ms/batch
Step 00017200 L_out=0.069 Acc=0.86 ---247120.58 ms/batch
Step 00017250 L_out=0.080 Acc=0.91 ---270406.36 ms/batch
Step 00017300 L_out=0.199 Acc=0.85 ---293734.72 ms/batch
Round 2 | epoch=25, train costTime=313.4580194950104, | total_activation_sum=12214026
Round 2 | epoch=26, current m_iou=0.5850397433343444
Round 2 | Best m_IoU is: 0.585, OA is: 0.883 | val costTime=116.89824652671814
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.166 Acc=0.95 --- 3831.03 ms/batch
Step 00017400 L_out=0.130 Acc=0.85 ---27437.67 ms/batch
Step 00017450 L_out=0.120 Acc=0.91 ---51110.45 ms/batch
Step 00017500 L_out=0.036 Acc=0.89 ---74616.51 ms/batch
Step 00017550 L_out=0.065 Acc=0.77 ---98230.48 ms/batch
Step 00017600 L_out=0.059 Acc=0.88 ---121895.61 ms/batch
Step 00017650 L_out=0.063 Acc=0.90 ---145655.26 ms/batch
Step 00017700 L_out=0.114 Acc=0.93 ---169247.64 ms/batch
Step 00017750 L_out=0.075 Acc=0.97 ---192740.46 ms/batch
Step 00017800 L_out=0.029 Acc=0.85 ---216018.24 ms/batch
Step 00017850 L_out=0.231 Acc=0.86 ---239543.61 ms/batch
Step 00017900 L_out=0.068 Acc=0.89 ---263006.03 ms/batch
Step 00017950 L_out=0.249 Acc=0.87 ---286711.63 ms/batch
Step 00018000 L_out=0.176 Acc=0.85 ---310093.99 ms/batch
Round 2 | epoch=26, train costTime=314.3829264640808, | total_activation_sum=11992205
Round 2 | epoch=27, current m_iou=0.5865889173164345
Round 2 | Best m_IoU is: 0.587, OA is: 0.884 | val costTime=117.33888268470764
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.089 Acc=0.92 ---19231.80 ms/batch
Step 00018100 L_out=0.114 Acc=0.91 ---42756.36 ms/batch
Step 00018150 L_out=0.229 Acc=0.85 ---66394.53 ms/batch
Step 00018200 L_out=0.062 Acc=0.90 ---90085.38 ms/batch
Step 00018250 L_out=0.155 Acc=0.92 ---113555.58 ms/batch
Step 00018300 L_out=0.099 Acc=0.87 ---137111.53 ms/batch
Step 00018350 L_out=0.090 Acc=0.95 ---160400.66 ms/batch
Step 00018400 L_out=0.077 Acc=0.90 ---183640.64 ms/batch
Step 00018450 L_out=0.047 Acc=0.83 ---207005.01 ms/batch
Step 00018500 L_out=0.109 Acc=0.88 ---230497.79 ms/batch
Step 00018550 L_out=0.110 Acc=0.93 ---254264.24 ms/batch
Step 00018600 L_out=0.191 Acc=0.86 ---277764.18 ms/batch
Step 00018650 L_out=0.140 Acc=0.83 ---301477.77 ms/batch
Round 2 | epoch=27, train costTime=313.59127974510193, | total_activation_sum=11987554
Round 2 | epoch=28, current m_iou=0.5912722466057467
Round 2 | Best m_IoU is: 0.591, OA is: 0.888 | val costTime=117.8558988571167
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.211 Acc=0.87 ---11376.57 ms/batch
Step 00018750 L_out=0.078 Acc=0.85 ---34840.21 ms/batch
Step 00018800 L_out=0.193 Acc=0.88 ---58518.60 ms/batch
Step 00018850 L_out=0.016 Acc=0.89 ---82072.62 ms/batch
Step 00018900 L_out=0.070 Acc=0.75 ---105479.92 ms/batch
Step 00018950 L_out=0.128 Acc=0.90 ---129224.76 ms/batch
Step 00019000 L_out=0.103 Acc=0.86 ---152796.46 ms/batch
Step 00019050 L_out=0.032 Acc=0.86 ---176221.90 ms/batch
Step 00019100 L_out=0.064 Acc=0.58 ---199633.82 ms/batch
Step 00019150 L_out=0.069 Acc=0.92 ---223348.77 ms/batch
Step 00019200 L_out=0.190 Acc=0.90 ---246829.91 ms/batch
Step 00019250 L_out=0.087 Acc=0.83 ---270530.26 ms/batch
Step 00019300 L_out=0.312 Acc=0.86 ---294058.02 ms/batch
Round 2 | epoch=28, train costTime=314.2177700996399, | total_activation_sum=11994555
Round 2 | epoch=29, current m_iou=0.5995061027267057
Round 2 | Best m_IoU is: 0.600, OA is: 0.885 | val costTime=117.39114809036255
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.034 Acc=0.68 --- 3259.64 ms/batch
Step 00019400 L_out=0.110 Acc=0.89 ---26823.26 ms/batch
Step 00019450 L_out=0.079 Acc=0.86 ---50564.75 ms/batch
Step 00019500 L_out=0.111 Acc=0.95 ---74275.83 ms/batch
Step 00019550 L_out=0.107 Acc=0.91 ---97767.08 ms/batch
Step 00019600 L_out=0.058 Acc=0.87 ---121375.42 ms/batch
Step 00019650 L_out=0.106 Acc=0.70 ---144735.24 ms/batch
Step 00019700 L_out=0.213 Acc=0.87 ---168238.56 ms/batch
Step 00019750 L_out=0.032 Acc=0.89 ---191880.94 ms/batch
Step 00019800 L_out=0.085 Acc=0.90 ---215288.78 ms/batch
Step 00019850 L_out=0.050 Acc=0.92 ---238638.51 ms/batch
Step 00019900 L_out=0.057 Acc=0.84 ---261977.58 ms/batch
Step 00019950 L_out=0.055 Acc=0.89 ---285582.51 ms/batch
Step 00020000 L_out=0.068 Acc=0.88 ---309236.94 ms/batch
Round 2 | epoch=29, train costTime=313.9096531867981, | total_activation_sum=11940843
Round 2 | epoch=30, current m_iou=0.6037927918162339
Round 2 | Best m_IoU is: 0.604, OA is: 0.888 | val costTime=117.53955507278442
Round 2 | ****EPOCH 30****
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.446 Acc=0.57 ---25615.41 ms/batch
Step 00000100 L_out=0.348 Acc=0.34 ---49436.62 ms/batch
Step 00000150 L_out=0.737 Acc=0.33 ---73164.73 ms/batch
Step 00000200 L_out=1.097 Acc=0.29 ---97009.19 ms/batch
Step 00000250 L_out=0.417 Acc=0.49 ---120669.89 ms/batch
Step 00000300 L_out=0.536 Acc=0.42 ---144613.73 ms/batch
Step 00000350 L_out=0.257 Acc=0.52 ---168501.78 ms/batch
Step 00000400 L_out=0.182 Acc=0.31 ---192136.39 ms/batch
Step 00000450 L_out=0.630 Acc=0.57 ---215907.33 ms/batch
Step 00000500 L_out=0.308 Acc=0.52 ---239456.44 ms/batch
Step 00000550 L_out=0.229 Acc=0.50 ---262892.50 ms/batch
Step 00000600 L_out=0.218 Acc=0.52 ---286767.87 ms/batch
Step 00000650 L_out=0.532 Acc=0.60 ---310355.09 ms/batch
Round 2 | epoch=0, train costTime=318.45234751701355, | total_activation_sum=9317671
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.354 Acc=0.48 ---15573.90 ms/batch
Step 00000750 L_out=0.186 Acc=0.62 ---39164.75 ms/batch
Step 00000800 L_out=0.377 Acc=0.42 ---62699.20 ms/batch
Step 00000850 L_out=0.520 Acc=0.69 ---86066.91 ms/batch
Step 00000900 L_out=0.454 Acc=0.60 ---109876.92 ms/batch
Step 00000950 L_out=0.662 Acc=0.45 ---133548.63 ms/batch
Step 00001000 L_out=0.258 Acc=0.63 ---157255.07 ms/batch
Step 00001050 L_out=0.421 Acc=0.64 ---180889.58 ms/batch
Step 00001100 L_out=0.290 Acc=0.32 ---204564.45 ms/batch
Step 00001150 L_out=0.352 Acc=0.69 ---228272.71 ms/batch
Step 00001200 L_out=0.545 Acc=0.75 ---251774.74 ms/batch
Step 00001250 L_out=0.281 Acc=0.35 ---275318.32 ms/batch
Step 00001300 L_out=0.466 Acc=0.56 ---299060.63 ms/batch
Round 2 | epoch=1, train costTime=315.12229466438293, | total_activation_sum=7926618
Round 2 | ****EPOCH 2****
Step 00001350 L_out=0.610 Acc=0.63 --- 7497.29 ms/batch
Step 00001400 L_out=0.035 Acc=0.58 ---31160.07 ms/batch
Step 00001450 L_out=0.323 Acc=0.53 ---54796.68 ms/batch
Step 00001500 L_out=0.350 Acc=0.66 ---78512.70 ms/batch
Step 00001550 L_out=0.207 Acc=0.62 ---102073.99 ms/batch
Step 00001600 L_out=0.371 Acc=0.46 ---125601.59 ms/batch
Step 00001650 L_out=0.281 Acc=0.67 ---149180.30 ms/batch
Step 00001700 L_out=0.240 Acc=0.69 ---172923.13 ms/batch
Step 00001750 L_out=0.356 Acc=0.71 ---196626.80 ms/batch
Step 00001800 L_out=0.282 Acc=0.45 ---219998.82 ms/batch
Step 00001850 L_out=0.257 Acc=0.77 ---243553.64 ms/batch
Step 00001900 L_out=0.207 Acc=0.69 ---267296.82 ms/batch
Step 00001950 L_out=0.147 Acc=0.76 ---290813.86 ms/batch
Step 00002000 L_out=0.249 Acc=0.62 ---314272.29 ms/batch
Round 2 | epoch=2, train costTime=314.750364780426, | total_activation_sum=7763969
Round 2 | ****EPOCH 3****
Step 00002050 L_out=0.131 Acc=0.63 ---23168.77 ms/batch
Step 00002100 L_out=0.297 Acc=0.49 ---46858.67 ms/batch
Step 00002150 L_out=0.296 Acc=0.52 ---70452.54 ms/batch
Step 00002200 L_out=0.298 Acc=0.49 ---93956.18 ms/batch
Step 00002250 L_out=0.343 Acc=0.62 ---117351.89 ms/batch
Step 00002300 L_out=0.531 Acc=0.58 ---141208.24 ms/batch
Step 00002350 L_out=0.794 Acc=0.55 ---164804.09 ms/batch
Step 00002400 L_out=0.242 Acc=0.77 ---188549.17 ms/batch
Step 00002450 L_out=0.479 Acc=0.73 ---212139.87 ms/batch
Step 00002500 L_out=0.220 Acc=0.69 ---235741.52 ms/batch
Step 00002550 L_out=0.690 Acc=0.64 ---259296.35 ms/batch
Step 00002600 L_out=0.023 Acc=0.54 ---283008.23 ms/batch
Step 00002650 L_out=0.217 Acc=0.58 ---306665.50 ms/batch
Round 2 | epoch=3, train costTime=315.19312858581543, | total_activation_sum=7889405
Round 2 | ****EPOCH 4****
Step 00002700 L_out=0.063 Acc=0.65 ---15078.54 ms/batch
Step 00002750 L_out=0.277 Acc=0.82 ---38558.34 ms/batch
Step 00002800 L_out=0.296 Acc=0.55 ---62207.07 ms/batch
Step 00002850 L_out=0.423 Acc=0.62 ---85914.28 ms/batch
Step 00002900 L_out=0.476 Acc=0.67 ---109471.83 ms/batch
Step 00002950 L_out=0.492 Acc=0.67 ---133162.11 ms/batch
Step 00003000 L_out=0.206 Acc=0.73 ---156739.60 ms/batch
Step 00003050 L_out=0.251 Acc=0.20 ---180323.04 ms/batch
Step 00003100 L_out=0.111 Acc=0.55 ---203777.63 ms/batch
Step 00003150 L_out=0.267 Acc=0.53 ---227348.52 ms/batch
Step 00003200 L_out=0.278 Acc=0.74 ---251148.91 ms/batch
Step 00003250 L_out=1.217 Acc=0.51 ---274542.53 ms/batch
Step 00003300 L_out=0.440 Acc=0.53 ---298029.15 ms/batch
Round 2 | epoch=4, train costTime=314.6576681137085, | total_activation_sum=7956274
Round 2 | ****EPOCH 5****
Step 00003350 L_out=0.289 Acc=0.57 --- 7219.63 ms/batch
Step 00003400 L_out=0.432 Acc=0.50 ---31220.16 ms/batch
Step 00003450 L_out=0.354 Acc=0.59 ---54840.16 ms/batch
Step 00003500 L_out=0.348 Acc=0.82 ---78444.44 ms/batch
Step 00003550 L_out=0.366 Acc=0.57 ---102089.31 ms/batch
Step 00003600 L_out=0.115 Acc=0.67 ---125650.58 ms/batch
Step 00003650 L_out=0.373 Acc=0.61 ---149281.34 ms/batch
Step 00003700 L_out=0.153 Acc=0.67 ---172607.90 ms/batch
Step 00003750 L_out=0.161 Acc=0.72 ---196208.34 ms/batch
Step 00003800 L_out=0.213 Acc=0.62 ---219995.18 ms/batch
Step 00003850 L_out=0.204 Acc=0.49 ---243805.91 ms/batch
Step 00003900 L_out=0.158 Acc=0.52 ---267437.28 ms/batch
Step 00003950 L_out=0.138 Acc=0.66 ---291049.71 ms/batch
Step 00004000 L_out=0.338 Acc=0.70 ---314904.09 ms/batch
Round 2 | epoch=5, train costTime=315.8819274902344, | total_activation_sum=7863844
Round 2 | ****EPOCH 6****
Step 00004050 L_out=0.354 Acc=0.82 ---22915.56 ms/batch
Step 00004100 L_out=0.235 Acc=0.51 ---46592.03 ms/batch
Step 00004150 L_out=0.205 Acc=0.69 ---70295.09 ms/batch
Step 00004200 L_out=0.207 Acc=0.78 ---93968.57 ms/batch
Step 00004250 L_out=0.362 Acc=0.68 ---117497.83 ms/batch
Step 00004300 L_out=0.098 Acc=0.62 ---141251.91 ms/batch
Step 00004350 L_out=0.191 Acc=0.65 ---164854.99 ms/batch
Step 00004400 L_out=0.197 Acc=0.86 ---188501.21 ms/batch
Step 00004450 L_out=0.302 Acc=0.50 ---212037.53 ms/batch
Step 00004500 L_out=0.122 Acc=0.60 ---235583.59 ms/batch
Step 00004550 L_out=0.195 Acc=0.56 ---259022.65 ms/batch
Step 00004600 L_out=0.053 Acc=0.75 ---282637.22 ms/batch
Step 00004650 L_out=0.156 Acc=0.85 ---306464.62 ms/batch
Round 2 | epoch=6, train costTime=315.4442057609558, | total_activation_sum=7925983
Round 2 | ****EPOCH 7****
Step 00004700 L_out=0.242 Acc=0.78 ---14600.00 ms/batch
Step 00004750 L_out=0.103 Acc=0.83 ---38112.68 ms/batch
Step 00004800 L_out=0.260 Acc=0.74 ---61579.78 ms/batch
Step 00004850 L_out=0.152 Acc=0.70 ---85071.33 ms/batch
Step 00004900 L_out=0.073 Acc=0.75 ---108589.53 ms/batch
Step 00004950 L_out=0.216 Acc=0.76 ---132379.44 ms/batch
Step 00005000 L_out=0.258 Acc=0.74 ---156102.66 ms/batch
Step 00005050 L_out=0.157 Acc=0.74 ---179570.25 ms/batch
Step 00005100 L_out=0.083 Acc=0.64 ---203169.33 ms/batch
Step 00005150 L_out=0.303 Acc=0.64 ---226794.19 ms/batch
Step 00005200 L_out=0.170 Acc=0.92 ---250431.12 ms/batch
Step 00005250 L_out=0.378 Acc=0.78 ---274199.55 ms/batch
Step 00005300 L_out=0.190 Acc=0.81 ---297873.76 ms/batch
Round 2 | epoch=7, train costTime=314.70400524139404, | total_activation_sum=7919552
Round 2 | ****EPOCH 8****
Step 00005350 L_out=0.324 Acc=0.65 --- 6637.30 ms/batch
Step 00005400 L_out=0.101 Acc=0.57 ---30050.58 ms/batch
Step 00005450 L_out=0.226 Acc=0.73 ---53642.76 ms/batch
Step 00005500 L_out=0.494 Acc=0.73 ---77067.39 ms/batch
Step 00005550 L_out=0.077 Acc=0.85 ---100749.90 ms/batch
Step 00005600 L_out=0.383 Acc=0.59 ---124354.77 ms/batch
Step 00005650 L_out=0.400 Acc=0.73 ---148085.19 ms/batch
Step 00005700 L_out=0.124 Acc=0.77 ---171866.57 ms/batch
Step 00005750 L_out=0.061 Acc=0.81 ---195485.12 ms/batch
Step 00005800 L_out=0.184 Acc=0.72 ---218995.14 ms/batch
Step 00005850 L_out=0.583 Acc=0.71 ---242628.85 ms/batch
Step 00005900 L_out=0.112 Acc=0.82 ---266265.71 ms/batch
Step 00005950 L_out=0.134 Acc=0.75 ---289886.40 ms/batch
Step 00006000 L_out=0.164 Acc=0.60 ---313815.00 ms/batch
Round 2 | epoch=8, train costTime=315.2126097679138, | total_activation_sum=7929281
Round 2 | ****EPOCH 9****
Step 00006050 L_out=0.109 Acc=0.77 ---22106.42 ms/batch
Step 00006100 L_out=0.162 Acc=0.75 ---45673.22 ms/batch
Step 00006150 L_out=0.165 Acc=0.74 ---69271.30 ms/batch
Step 00006200 L_out=0.197 Acc=0.77 ---92962.60 ms/batch
Step 00006250 L_out=0.290 Acc=0.82 ---116524.49 ms/batch
Step 00006300 L_out=0.136 Acc=0.53 ---140111.56 ms/batch
Step 00006350 L_out=0.121 Acc=0.86 ---163775.79 ms/batch
Step 00006400 L_out=0.039 Acc=0.69 ---187553.24 ms/batch
Step 00006450 L_out=0.101 Acc=0.83 ---211045.88 ms/batch
Step 00006500 L_out=0.042 Acc=0.75 ---234611.80 ms/batch
Step 00006550 L_out=0.169 Acc=0.83 ---258281.05 ms/batch
Step 00006600 L_out=0.147 Acc=0.68 ---281985.85 ms/batch
Step 00006650 L_out=0.421 Acc=0.93 ---305606.35 ms/batch
Round 2 | epoch=9, train costTime=315.1604092121124, | total_activation_sum=8018534
Round 2 | ****EPOCH 10****
Step 00006700 L_out=0.329 Acc=0.64 ---14098.49 ms/batch
Step 00006750 L_out=0.124 Acc=0.70 ---37576.89 ms/batch
Step 00006800 L_out=0.127 Acc=0.71 ---61204.41 ms/batch
Step 00006850 L_out=0.076 Acc=0.86 ---84647.00 ms/batch
Step 00006900 L_out=0.268 Acc=0.74 ---108107.18 ms/batch
Step 00006950 L_out=0.241 Acc=0.80 ---131617.95 ms/batch
Step 00007000 L_out=0.293 Acc=0.85 ---155342.42 ms/batch
Step 00007050 L_out=0.160 Acc=0.76 ---178870.34 ms/batch
Step 00007100 L_out=0.289 Acc=0.73 ---202365.82 ms/batch
Step 00007150 L_out=0.096 Acc=0.85 ---225898.64 ms/batch
Step 00007200 L_out=0.077 Acc=0.75 ---249554.15 ms/batch
Step 00007250 L_out=0.246 Acc=0.72 ---273154.13 ms/batch
Step 00007300 L_out=0.162 Acc=0.66 ---297149.19 ms/batch
Round 2 | epoch=10, train costTime=314.74740386009216, | total_activation_sum=7891649
Round 2 | ****EPOCH 11****
Step 00007350 L_out=0.087 Acc=0.82 --- 6131.86 ms/batch
Step 00007400 L_out=0.221 Acc=0.73 ---29612.34 ms/batch
Step 00007450 L_out=0.245 Acc=0.84 ---53591.21 ms/batch
Step 00007500 L_out=0.079 Acc=0.81 ---77313.87 ms/batch
Step 00007550 L_out=0.048 Acc=0.78 ---100745.14 ms/batch
Step 00007600 L_out=0.178 Acc=0.84 ---124134.34 ms/batch
Step 00007650 L_out=0.145 Acc=0.74 ---147692.58 ms/batch
Step 00007700 L_out=0.095 Acc=0.68 ---171200.31 ms/batch
Step 00007750 L_out=0.129 Acc=0.83 ---194915.21 ms/batch
Step 00007800 L_out=1.287 Acc=0.72 ---218466.65 ms/batch
Step 00007850 L_out=0.077 Acc=0.83 ---241842.45 ms/batch
Step 00007900 L_out=0.210 Acc=0.78 ---265627.60 ms/batch
Step 00007950 L_out=0.229 Acc=0.82 ---289343.83 ms/batch
Step 00008000 L_out=0.117 Acc=0.86 ---312828.44 ms/batch
Round 2 | epoch=11, train costTime=314.7153363227844, | total_activation_sum=8067431
Round 2 | ****EPOCH 12****
Step 00008050 L_out=0.215 Acc=0.81 ---21812.83 ms/batch
Step 00008100 L_out=0.119 Acc=0.88 ---45560.87 ms/batch
Step 00008150 L_out=0.252 Acc=0.78 ---69150.09 ms/batch
Step 00008200 L_out=0.178 Acc=0.81 ---92722.48 ms/batch
Step 00008250 L_out=0.155 Acc=0.78 ---116536.04 ms/batch
Step 00008300 L_out=0.140 Acc=0.79 ---140090.24 ms/batch
Step 00008350 L_out=0.051 Acc=0.79 ---163520.95 ms/batch
Step 00008400 L_out=0.227 Acc=0.77 ---187120.07 ms/batch
Step 00008450 L_out=0.050 Acc=0.82 ---210547.21 ms/batch
Step 00008500 L_out=0.204 Acc=0.78 ---234146.10 ms/batch
Step 00008550 L_out=0.231 Acc=0.64 ---257735.98 ms/batch
Step 00008600 L_out=0.103 Acc=0.85 ---281234.14 ms/batch
Step 00008650 L_out=0.043 Acc=0.87 ---304683.05 ms/batch
Round 2 | epoch=12, train costTime=314.5931875705719, | total_activation_sum=7795534
Round 2 | ****EPOCH 13****
Step 00008700 L_out=0.140 Acc=0.84 ---13800.59 ms/batch
Step 00008750 L_out=0.295 Acc=0.81 ---37304.47 ms/batch
Step 00008800 L_out=0.034 Acc=0.89 ---61078.79 ms/batch
Step 00008850 L_out=0.129 Acc=0.71 ---84943.48 ms/batch
Step 00008900 L_out=0.103 Acc=0.78 ---108542.05 ms/batch
Step 00008950 L_out=0.049 Acc=0.84 ---131924.76 ms/batch
Step 00009000 L_out=0.188 Acc=0.79 ---155629.71 ms/batch
Step 00009050 L_out=0.152 Acc=0.85 ---179567.53 ms/batch
Step 00009100 L_out=0.158 Acc=0.80 ---203252.35 ms/batch
Step 00009150 L_out=0.193 Acc=0.80 ---226878.45 ms/batch
Step 00009200 L_out=0.150 Acc=0.85 ---250393.84 ms/batch
Step 00009250 L_out=0.113 Acc=0.73 ---273979.13 ms/batch
Step 00009300 L_out=0.400 Acc=0.79 ---297592.52 ms/batch
Round 2 | epoch=13, train costTime=315.6220633983612, | total_activation_sum=7907647
Round 2 | ****EPOCH 14****
Step 00009350 L_out=0.199 Acc=0.81 --- 5594.25 ms/batch
Step 00009400 L_out=0.240 Acc=0.84 ---29082.66 ms/batch
Step 00009450 L_out=0.333 Acc=0.85 ---52542.74 ms/batch
Step 00009500 L_out=0.167 Acc=0.88 ---76264.70 ms/batch
Step 00009550 L_out=0.092 Acc=0.79 ---99965.72 ms/batch
Step 00009600 L_out=0.150 Acc=0.79 ---123748.56 ms/batch
Step 00009650 L_out=0.231 Acc=0.80 ---147442.98 ms/batch
Step 00009700 L_out=0.117 Acc=0.77 ---171086.19 ms/batch
Step 00009750 L_out=0.116 Acc=0.81 ---194631.16 ms/batch
Step 00009800 L_out=0.486 Acc=0.83 ---218193.99 ms/batch
Step 00009850 L_out=0.017 Acc=0.82 ---241789.70 ms/batch
Step 00009900 L_out=0.164 Acc=0.79 ---265420.47 ms/batch
Step 00009950 L_out=0.009 Acc=0.85 ---289138.34 ms/batch
Step 00010000 L_out=0.229 Acc=0.84 ---312759.18 ms/batch
Round 2 | epoch=14, train costTime=315.0709648132324, | total_activation_sum=7864952
Round 2 | ****EPOCH 15****
Step 00010050 L_out=0.214 Acc=0.67 ---21246.52 ms/batch
Step 00010100 L_out=0.086 Acc=0.91 ---44560.32 ms/batch
Step 00010150 L_out=0.090 Acc=0.85 ---68342.81 ms/batch
Step 00010200 L_out=0.271 Acc=0.79 ---91912.65 ms/batch
Step 00010250 L_out=0.283 Acc=0.83 ---115410.20 ms/batch
Step 00010300 L_out=0.443 Acc=0.81 ---139034.74 ms/batch
Step 00010350 L_out=0.262 Acc=0.78 ---162575.53 ms/batch
Step 00010400 L_out=0.133 Acc=0.81 ---186202.53 ms/batch
Step 00010450 L_out=0.180 Acc=0.68 ---209857.33 ms/batch
Step 00010500 L_out=0.239 Acc=0.78 ---233437.43 ms/batch
Step 00010550 L_out=0.308 Acc=0.71 ---257073.68 ms/batch
Step 00010600 L_out=0.062 Acc=0.87 ---280339.34 ms/batch
Step 00010650 L_out=0.041 Acc=0.88 ---304005.56 ms/batch
Round 2 | epoch=15, train costTime=314.5180561542511, | total_activation_sum=7997795
Round 2 | ****EPOCH 16****
Step 00010700 L_out=0.254 Acc=0.87 ---13156.43 ms/batch
Step 00010750 L_out=0.111 Acc=0.87 ---36612.80 ms/batch
Step 00010800 L_out=0.051 Acc=0.87 ---60201.61 ms/batch
Step 00010850 L_out=0.087 Acc=0.90 ---83721.75 ms/batch
Step 00010900 L_out=0.113 Acc=0.85 ---107208.50 ms/batch
Step 00010950 L_out=0.050 Acc=0.89 ---130741.78 ms/batch
Step 00011000 L_out=0.066 Acc=0.81 ---154374.88 ms/batch
Step 00011050 L_out=0.293 Acc=0.64 ---177960.44 ms/batch
Step 00011100 L_out=0.112 Acc=0.85 ---201671.50 ms/batch
Step 00011150 L_out=0.033 Acc=0.87 ---225253.90 ms/batch
Step 00011200 L_out=0.156 Acc=0.88 ---248842.13 ms/batch
Step 00011250 L_out=0.106 Acc=0.84 ---272489.74 ms/batch
Step 00011300 L_out=0.167 Acc=0.83 ---296127.09 ms/batch
Round 2 | epoch=16, train costTime=314.53819394111633, | total_activation_sum=7935787
Round 2 | ****EPOCH 17****
Step 00011350 L_out=0.089 Acc=0.86 --- 5182.17 ms/batch
Step 00011400 L_out=0.133 Acc=0.92 ---28888.09 ms/batch
Step 00011450 L_out=0.141 Acc=0.75 ---52751.12 ms/batch
Step 00011500 L_out=0.116 Acc=0.88 ---76581.58 ms/batch
Step 00011550 L_out=0.029 Acc=0.95 ---99997.99 ms/batch
Step 00011600 L_out=0.041 Acc=0.73 ---123607.61 ms/batch
Step 00011650 L_out=0.125 Acc=0.75 ---147196.46 ms/batch
Step 00011700 L_out=0.102 Acc=0.83 ---170873.81 ms/batch
Step 00011750 L_out=0.099 Acc=0.79 ---194339.23 ms/batch
Step 00011800 L_out=0.078 Acc=0.86 ---217683.47 ms/batch
Step 00011850 L_out=0.121 Acc=0.84 ---241332.46 ms/batch
Step 00011900 L_out=0.084 Acc=0.84 ---264832.12 ms/batch
Step 00011950 L_out=0.160 Acc=0.87 ---288356.74 ms/batch
Step 00012000 L_out=0.026 Acc=0.88 ---311978.56 ms/batch
Round 2 | epoch=17, train costTime=314.75844073295593, | total_activation_sum=7920175
Round 2 | epoch=18, current m_iou=0.560905906857103
Round 2 | Best m_IoU is: 0.561, OA is: 0.880 | val costTime=117.46338820457458
Round 2 | ****EPOCH 18****
Step 00012050 L_out=0.037 Acc=0.94 ---20724.72 ms/batch
Step 00012100 L_out=0.072 Acc=0.82 ---44303.47 ms/batch
Step 00012150 L_out=0.048 Acc=0.82 ---67828.60 ms/batch
Step 00012200 L_out=0.113 Acc=0.71 ---91302.46 ms/batch
Step 00012250 L_out=0.098 Acc=0.81 ---115003.71 ms/batch
Step 00012300 L_out=0.214 Acc=0.85 ---138626.46 ms/batch
Step 00012350 L_out=0.096 Acc=0.91 ---162168.23 ms/batch
Step 00012400 L_out=0.052 Acc=0.86 ---185741.52 ms/batch
Step 00012450 L_out=0.073 Acc=0.71 ---209365.41 ms/batch
Step 00012500 L_out=0.099 Acc=0.84 ---232892.84 ms/batch
Step 00012550 L_out=0.142 Acc=0.81 ---256540.61 ms/batch
Step 00012600 L_out=0.120 Acc=0.80 ---280079.08 ms/batch
Step 00012650 L_out=0.063 Acc=0.87 ---303788.52 ms/batch
Round 2 | epoch=18, train costTime=314.6614201068878, | total_activation_sum=7875183
Round 2 | epoch=19, current m_iou=0.5507788694249142
Round 2 | Best m_IoU is: 0.561, OA is: 0.880 | val costTime=117.12477397918701
Round 2 | ****EPOCH 19****
Step 00012700 L_out=0.315 Acc=0.88 ---12692.98 ms/batch
Step 00012750 L_out=0.051 Acc=0.88 ---36185.52 ms/batch
Step 00012800 L_out=0.170 Acc=0.87 ---59950.75 ms/batch
Step 00012850 L_out=0.068 Acc=0.83 ---83294.40 ms/batch
Step 00012900 L_out=0.260 Acc=0.78 ---106950.37 ms/batch
Step 00012950 L_out=0.115 Acc=0.77 ---130595.86 ms/batch
Step 00013000 L_out=0.094 Acc=0.93 ---154416.96 ms/batch
Step 00013050 L_out=0.106 Acc=0.85 ---178029.27 ms/batch
Step 00013100 L_out=0.019 Acc=0.90 ---201442.23 ms/batch
Step 00013150 L_out=0.041 Acc=0.83 ---224999.97 ms/batch
Step 00013200 L_out=0.116 Acc=0.86 ---248516.97 ms/batch
Step 00013250 L_out=0.240 Acc=0.83 ---272121.68 ms/batch
Step 00013300 L_out=0.245 Acc=0.79 ---295651.26 ms/batch
Round 2 | epoch=19, train costTime=314.4588749408722, | total_activation_sum=7936310
Round 2 | epoch=20, current m_iou=0.5943113784590887
Round 2 | Best m_IoU is: 0.594, OA is: 0.885 | val costTime=116.79181408882141
Round 2 | ****EPOCH 20****
Step 00013350 L_out=0.121 Acc=0.79 --- 4687.69 ms/batch
Step 00013400 L_out=0.140 Acc=0.80 ---28228.26 ms/batch
Step 00013450 L_out=0.153 Acc=0.78 ---51867.60 ms/batch
Step 00013500 L_out=0.271 Acc=0.76 ---75522.12 ms/batch
Step 00013550 L_out=0.045 Acc=0.88 ---99370.43 ms/batch
Step 00013600 L_out=0.124 Acc=0.81 ---123199.73 ms/batch
Step 00013650 L_out=0.014 Acc=0.81 ---146846.11 ms/batch
Step 00013700 L_out=0.038 Acc=0.90 ---170490.11 ms/batch
Step 00013750 L_out=0.238 Acc=0.73 ---194028.86 ms/batch
Step 00013800 L_out=0.126 Acc=0.90 ---217648.73 ms/batch
Step 00013850 L_out=0.187 Acc=0.84 ---241385.24 ms/batch
Step 00013900 L_out=0.132 Acc=0.88 ---264948.17 ms/batch
Step 00013950 L_out=0.098 Acc=0.86 ---288742.41 ms/batch
Step 00014000 L_out=0.160 Acc=0.79 ---312316.52 ms/batch
Round 2 | epoch=20, train costTime=315.59909558296204, | total_activation_sum=7986130
Round 2 | epoch=21, current m_iou=0.6023549092196745
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=117.27460718154907
Round 2 | ****EPOCH 21****
Step 00014050 L_out=0.024 Acc=0.89 ---20193.74 ms/batch
Step 00014100 L_out=0.053 Acc=0.89 ---43697.18 ms/batch
Step 00014150 L_out=0.362 Acc=0.82 ---67353.65 ms/batch
Step 00014200 L_out=0.138 Acc=0.73 ---90809.32 ms/batch
Step 00014250 L_out=0.179 Acc=0.83 ---114495.93 ms/batch
Step 00014300 L_out=0.105 Acc=0.69 ---138179.43 ms/batch
Step 00014350 L_out=0.164 Acc=0.75 ---161800.56 ms/batch
Step 00014400 L_out=0.181 Acc=0.84 ---185092.89 ms/batch
Step 00014450 L_out=0.077 Acc=0.85 ---208699.48 ms/batch
Step 00014500 L_out=0.127 Acc=0.81 ---232407.48 ms/batch
Step 00014550 L_out=0.112 Acc=0.86 ---255835.66 ms/batch
Step 00014600 L_out=0.093 Acc=0.87 ---279580.82 ms/batch
Step 00014650 L_out=0.090 Acc=0.86 ---302886.22 ms/batch
Round 2 | epoch=21, train costTime=314.2630977630615, | total_activation_sum=7874294
Round 2 | epoch=22, current m_iou=0.5819814678858986
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.81492853164673
Round 2 | ****EPOCH 22****
Step 00014700 L_out=0.080 Acc=0.85 ---12210.78 ms/batch
Step 00014750 L_out=0.073 Acc=0.86 ---35919.68 ms/batch
Step 00014800 L_out=0.054 Acc=0.87 ---59706.32 ms/batch
Step 00014850 L_out=0.124 Acc=0.84 ---83465.85 ms/batch
Step 00014900 L_out=0.053 Acc=0.87 ---107043.21 ms/batch
Step 00014950 L_out=0.113 Acc=0.82 ---130734.18 ms/batch
Step 00015000 L_out=0.065 Acc=0.78 ---154241.22 ms/batch
Step 00015050 L_out=0.101 Acc=0.91 ---177804.18 ms/batch
Step 00015100 L_out=0.060 Acc=0.85 ---201094.66 ms/batch
Step 00015150 L_out=0.155 Acc=0.78 ---224633.86 ms/batch
Step 00015200 L_out=0.126 Acc=0.84 ---248307.21 ms/batch
Step 00015250 L_out=0.092 Acc=0.83 ---271774.64 ms/batch
Step 00015300 L_out=0.121 Acc=0.89 ---295459.12 ms/batch
Round 2 | epoch=22, train costTime=315.0128791332245, | total_activation_sum=7982102
Round 2 | epoch=23, current m_iou=0.583729418906796
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.90910220146179
Round 2 | ****EPOCH 23****
Step 00015350 L_out=0.253 Acc=0.79 --- 4279.79 ms/batch
Step 00015400 L_out=0.057 Acc=0.80 ---28018.09 ms/batch
Step 00015450 L_out=0.060 Acc=0.86 ---51745.36 ms/batch
Step 00015500 L_out=0.091 Acc=0.78 ---75431.02 ms/batch
Step 00015550 L_out=0.082 Acc=0.93 ---99139.79 ms/batch
Step 00015600 L_out=0.145 Acc=0.76 ---122835.10 ms/batch
Step 00015650 L_out=0.039 Acc=0.92 ---146523.20 ms/batch
Step 00015700 L_out=0.106 Acc=0.89 ---169916.27 ms/batch
Step 00015750 L_out=0.066 Acc=0.85 ---193452.88 ms/batch
Step 00015800 L_out=0.022 Acc=0.92 ---216979.59 ms/batch
Step 00015850 L_out=0.072 Acc=0.90 ---240770.89 ms/batch
Step 00015900 L_out=0.076 Acc=0.78 ---263662.16 ms/batch
Step 00015950 L_out=0.064 Acc=0.83 ---287108.21 ms/batch
Step 00016000 L_out=0.017 Acc=0.64 ---310702.12 ms/batch
Round 2 | epoch=23, train costTime=314.41558599472046, | total_activation_sum=7965067
Round 2 | epoch=24, current m_iou=0.5842066710429483
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=117.19606471061707
Round 2 | ****EPOCH 24****
Step 00016050 L_out=0.041 Acc=0.93 ---19784.35 ms/batch
Step 00016100 L_out=0.150 Acc=0.88 ---43249.07 ms/batch
Step 00016150 L_out=0.093 Acc=0.81 ---66865.92 ms/batch
Step 00016200 L_out=0.040 Acc=0.86 ---90361.13 ms/batch
Step 00016250 L_out=0.311 Acc=0.80 ---113524.72 ms/batch
Step 00016300 L_out=0.051 Acc=0.85 ---136354.82 ms/batch
Step 00016350 L_out=0.028 Acc=0.90 ---160020.33 ms/batch
Step 00016400 L_out=0.160 Acc=0.86 ---183467.46 ms/batch
Step 00016450 L_out=0.151 Acc=0.88 ---206404.76 ms/batch
Step 00016500 L_out=0.107 Acc=0.92 ---229948.53 ms/batch
Step 00016550 L_out=0.082 Acc=0.92 ---253328.94 ms/batch
Step 00016600 L_out=0.054 Acc=0.87 ---276681.12 ms/batch
Step 00016650 L_out=0.081 Acc=0.87 ---300283.27 ms/batch
Round 2 | epoch=24, train costTime=312.0179524421692, | total_activation_sum=7865540
Round 2 | epoch=25, current m_iou=0.5918895194873515
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=117.1863420009613
Round 2 | ****EPOCH 25****
Step 00016700 L_out=0.067 Acc=0.83 ---11813.33 ms/batch
Step 00016750 L_out=0.044 Acc=0.87 ---35306.68 ms/batch
Step 00016800 L_out=0.037 Acc=0.84 ---59201.21 ms/batch
Step 00016850 L_out=0.073 Acc=0.88 ---82709.01 ms/batch
Step 00016900 L_out=0.120 Acc=0.87 ---106338.92 ms/batch
Step 00016950 L_out=0.145 Acc=0.87 ---129680.95 ms/batch
Step 00017000 L_out=0.038 Acc=0.93 ---153240.60 ms/batch
Step 00017050 L_out=0.063 Acc=0.88 ---176667.99 ms/batch
Step 00017100 L_out=0.106 Acc=0.88 ---200293.79 ms/batch
Step 00017150 L_out=0.374 Acc=0.80 ---224066.71 ms/batch
Step 00017200 L_out=0.056 Acc=0.86 ---247461.14 ms/batch
Step 00017250 L_out=0.142 Acc=0.89 ---271166.70 ms/batch
Step 00017300 L_out=0.091 Acc=0.83 ---294942.99 ms/batch
Round 2 | epoch=25, train costTime=314.9484646320343, | total_activation_sum=7945822
Round 2 | epoch=26, current m_iou=0.5944384170812516
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.82550764083862
Round 2 | ****EPOCH 26****
Step 00017350 L_out=0.028 Acc=0.89 --- 3795.10 ms/batch
Step 00017400 L_out=0.137 Acc=0.79 ---27258.02 ms/batch
Step 00017450 L_out=0.100 Acc=0.89 ---50872.27 ms/batch
Step 00017500 L_out=0.159 Acc=0.91 ---74349.94 ms/batch
Step 00017550 L_out=0.140 Acc=0.91 ---97842.72 ms/batch
Step 00017600 L_out=0.161 Acc=0.85 ---121390.19 ms/batch
Step 00017650 L_out=0.052 Acc=0.75 ---144971.35 ms/batch
Step 00017700 L_out=0.056 Acc=0.86 ---168885.28 ms/batch
Step 00017750 L_out=0.089 Acc=0.88 ---192645.20 ms/batch
Step 00017800 L_out=0.121 Acc=0.77 ---216083.23 ms/batch
Step 00017850 L_out=0.086 Acc=0.86 ---239611.70 ms/batch
Step 00017900 L_out=0.040 Acc=0.76 ---263468.85 ms/batch
Step 00017950 L_out=0.080 Acc=0.86 ---286959.71 ms/batch
Step 00018000 L_out=0.101 Acc=0.81 ---310488.12 ms/batch
Round 2 | epoch=26, train costTime=314.7818355560303, | total_activation_sum=7849015
Round 2 | epoch=27, current m_iou=0.5769697896600876
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.2762987613678
Round 2 | ****EPOCH 27****
Step 00018050 L_out=0.028 Acc=0.92 ---19250.51 ms/batch
Step 00018100 L_out=0.064 Acc=0.87 ---42550.88 ms/batch
Step 00018150 L_out=0.067 Acc=0.89 ---66005.46 ms/batch
Step 00018200 L_out=0.006 Acc=0.76 ---89464.14 ms/batch
Step 00018250 L_out=0.032 Acc=0.87 ---113144.21 ms/batch
Step 00018300 L_out=0.058 Acc=0.92 ---136506.83 ms/batch
Step 00018350 L_out=0.077 Acc=0.92 ---160006.80 ms/batch
Step 00018400 L_out=0.091 Acc=0.89 ---183532.45 ms/batch
Step 00018450 L_out=0.093 Acc=0.86 ---207165.58 ms/batch
Step 00018500 L_out=0.094 Acc=0.88 ---230652.04 ms/batch
Step 00018550 L_out=0.005 Acc=0.85 ---254282.89 ms/batch
Step 00018600 L_out=0.047 Acc=0.87 ---277845.93 ms/batch
Step 00018650 L_out=0.048 Acc=0.82 ---301388.28 ms/batch
Round 2 | epoch=27, train costTime=313.61257314682007, | total_activation_sum=8024489
Round 2 | epoch=28, current m_iou=0.5817298150491639
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.59222912788391
Round 2 | ****EPOCH 28****
Step 00018700 L_out=0.113 Acc=0.90 ---11305.35 ms/batch
Step 00018750 L_out=0.102 Acc=0.91 ---34816.18 ms/batch
Step 00018800 L_out=0.199 Acc=0.88 ---57755.47 ms/batch
Step 00018850 L_out=0.084 Acc=0.79 ---80772.14 ms/batch
Step 00018900 L_out=0.032 Acc=0.87 ---104457.10 ms/batch
Step 00018950 L_out=0.092 Acc=0.86 ---127962.79 ms/batch
Step 00019000 L_out=0.487 Acc=0.86 ---151599.93 ms/batch
Step 00019050 L_out=0.226 Acc=0.88 ---175097.72 ms/batch
Step 00019100 L_out=0.113 Acc=0.83 ---198661.76 ms/batch
Step 00019150 L_out=0.011 Acc=0.87 ---222188.35 ms/batch
Step 00019200 L_out=0.054 Acc=0.83 ---245589.93 ms/batch
Step 00019250 L_out=0.040 Acc=0.92 ---269381.69 ms/batch
Step 00019300 L_out=0.193 Acc=0.79 ---292997.82 ms/batch
Round 2 | epoch=28, train costTime=313.0715641975403, | total_activation_sum=7970984
Round 2 | epoch=29, current m_iou=0.5893856136709861
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.55952835083008
Round 2 | ****EPOCH 29****
Step 00019350 L_out=0.026 Acc=0.89 --- 3325.25 ms/batch
Step 00019400 L_out=0.042 Acc=0.86 ---27200.13 ms/batch
Step 00019450 L_out=0.226 Acc=0.82 ---50759.62 ms/batch
Step 00019500 L_out=0.030 Acc=0.85 ---74411.24 ms/batch
Step 00019550 L_out=0.022 Acc=0.90 ---97888.81 ms/batch
Step 00019600 L_out=0.101 Acc=0.88 ---121473.52 ms/batch
Step 00019650 L_out=0.038 Acc=0.87 ---145287.46 ms/batch
Step 00019700 L_out=0.052 Acc=0.88 ---168778.24 ms/batch
Step 00019750 L_out=0.064 Acc=0.84 ---192412.26 ms/batch
Step 00019800 L_out=0.159 Acc=0.84 ---216074.29 ms/batch
Step 00019850 L_out=0.038 Acc=0.88 ---239921.18 ms/batch
Step 00019900 L_out=0.085 Acc=0.90 ---263411.80 ms/batch
Step 00019950 L_out=0.060 Acc=0.93 ---286700.27 ms/batch
Step 00020000 L_out=0.010 Acc=0.92 ---310245.77 ms/batch
Round 2 | epoch=29, train costTime=314.96529936790466, | total_activation_sum=7871422
Round 2 | epoch=30, current m_iou=0.5854766172362027
Round 2 | Best m_IoU is: 0.602, OA is: 0.890 | val costTime=116.79225325584412
Round 2 | ****EPOCH 30****
Round 3 | ****EPOCH 0****
Step 00020050 L_out=0.597 Acc=0.72 ---19007.07 ms/batch
Step 00020100 L_out=1.642 Acc=0.64 ---42986.77 ms/batch
Step 00020150 L_out=0.422 Acc=0.79 ---66840.80 ms/batch
Step 00020200 L_out=1.021 Acc=0.56 ---90518.88 ms/batch
Step 00020250 L_out=0.271 Acc=0.45 ---114576.78 ms/batch
Step 00020300 L_out=0.629 Acc=0.73 ---138189.15 ms/batch
Step 00020350 L_out=0.447 Acc=0.61 ---162112.75 ms/batch
Step 00020400 L_out=0.267 Acc=0.63 ---185842.47 ms/batch
Step 00020450 L_out=1.484 Acc=0.79 ---209678.42 ms/batch
Step 00020500 L_out=0.393 Acc=0.77 ---233308.66 ms/batch
Step 00020550 L_out=0.451 Acc=0.65 ---256939.76 ms/batch
Step 00020600 L_out=0.381 Acc=0.81 ---280627.21 ms/batch
Step 00020650 L_out=0.694 Acc=0.80 ---304122.25 ms/batch
Round 3 | epoch=0, train costTime=317.1367254257202, | total_activation_sum=14871324
Round 3 | ****EPOCH 1****
Step 00020700 L_out=0.258 Acc=0.90 ---10903.82 ms/batch
Step 00020750 L_out=0.862 Acc=0.73 ---34694.18 ms/batch
Step 00020800 L_out=0.634 Acc=0.88 ---58258.34 ms/batch
Step 00020850 L_out=0.701 Acc=0.82 ---81910.97 ms/batch
Step 00020900 L_out=0.795 Acc=0.69 ---105586.48 ms/batch
Step 00020950 L_out=0.222 Acc=0.74 ---129139.76 ms/batch
Step 00021000 L_out=0.440 Acc=0.64 ---152655.31 ms/batch
Step 00021050 L_out=0.550 Acc=0.92 ---176269.34 ms/batch
Step 00021100 L_out=0.484 Acc=0.86 ---200015.73 ms/batch
Step 00021150 L_out=0.445 Acc=0.83 ---223688.39 ms/batch
Step 00021200 L_out=0.328 Acc=0.79 ---247324.89 ms/batch
Step 00021250 L_out=1.013 Acc=0.84 ---270935.24 ms/batch
Step 00021300 L_out=0.185 Acc=0.83 ---294788.51 ms/batch
Round 3 | epoch=1, train costTime=315.67206835746765, | total_activation_sum=13523106
Round 3 | ****EPOCH 2****
Step 00021350 L_out=0.169 Acc=0.68 --- 2844.01 ms/batch
Step 00021400 L_out=0.739 Acc=0.58 ---26420.93 ms/batch
Step 00021450 L_out=0.105 Acc=0.82 ---49971.49 ms/batch
Step 00021500 L_out=0.181 Acc=0.70 ---73684.67 ms/batch
Step 00021550 L_out=0.334 Acc=0.66 ---97409.62 ms/batch
Step 00021600 L_out=0.238 Acc=0.73 ---120926.93 ms/batch
Step 00021650 L_out=0.573 Acc=0.84 ---144470.53 ms/batch
Step 00021700 L_out=0.476 Acc=0.89 ---168258.08 ms/batch
Step 00021750 L_out=0.244 Acc=0.82 ---191960.85 ms/batch
Step 00021800 L_out=0.152 Acc=0.82 ---215540.77 ms/batch
Step 00021850 L_out=0.393 Acc=0.80 ---239274.42 ms/batch
Step 00021900 L_out=0.647 Acc=0.79 ---262828.31 ms/batch
Step 00021950 L_out=0.458 Acc=0.79 ---286546.92 ms/batch
Step 00022000 L_out=0.173 Acc=0.78 ---309965.26 ms/batch
Round 3 | epoch=2, train costTime=315.26182198524475, | total_activation_sum=13537848
Round 3 | ****EPOCH 3****
Step 00022050 L_out=0.202 Acc=0.90 ---18515.60 ms/batch
Step 00022100 L_out=0.413 Acc=0.66 ---42185.08 ms/batch
Step 00022150 L_out=0.160 Acc=0.66 ---65997.37 ms/batch
Step 00022200 L_out=0.557 Acc=0.87 ---89582.38 ms/batch
Step 00022250 L_out=0.284 Acc=0.76 ---113163.84 ms/batch
Step 00022300 L_out=0.331 Acc=0.67 ---136683.41 ms/batch
Step 00022350 L_out=0.085 Acc=0.89 ---160365.30 ms/batch
Step 00022400 L_out=0.276 Acc=0.76 ---184076.92 ms/batch
Step 00022450 L_out=0.405 Acc=0.71 ---207585.26 ms/batch
Step 00022500 L_out=0.138 Acc=0.84 ---231585.75 ms/batch
Step 00022550 L_out=0.311 Acc=0.87 ---255273.05 ms/batch
Step 00022600 L_out=0.533 Acc=0.81 ---279021.69 ms/batch
Step 00022650 L_out=0.551 Acc=0.83 ---302654.99 ms/batch
Round 3 | epoch=3, train costTime=315.9070818424225, | total_activation_sum=13683497
Round 3 | ****EPOCH 4****
Step 00022700 L_out=0.257 Acc=0.68 ---10342.09 ms/batch
Step 00022750 L_out=0.402 Acc=0.74 ---33555.75 ms/batch
Step 00022800 L_out=0.409 Acc=0.76 ---56645.81 ms/batch
Step 00022850 L_out=0.090 Acc=0.83 ---80286.03 ms/batch
Step 00022900 L_out=0.166 Acc=0.86 ---103669.99 ms/batch
Step 00022950 L_out=0.484 Acc=0.78 ---127475.30 ms/batch
Step 00023000 L_out=0.354 Acc=0.77 ---151138.85 ms/batch
Step 00023050 L_out=0.163 Acc=0.79 ---174972.25 ms/batch
Step 00023100 L_out=0.145 Acc=0.85 ---198175.86 ms/batch
Step 00023150 L_out=0.235 Acc=0.90 ---221685.56 ms/batch
Step 00023200 L_out=0.409 Acc=0.80 ---245226.36 ms/batch
Step 00023250 L_out=0.213 Acc=0.86 ---268819.70 ms/batch
Step 00023300 L_out=0.394 Acc=0.76 ---292542.38 ms/batch
Round 3 | epoch=4, train costTime=313.92569231987, | total_activation_sum=13570315
Round 3 | ****EPOCH 5****
Step 00023350 L_out=0.329 Acc=0.88 --- 2322.00 ms/batch
Step 00023400 L_out=0.190 Acc=0.78 ---25757.02 ms/batch
Step 00023450 L_out=0.539 Acc=0.85 ---49217.90 ms/batch
Step 00023500 L_out=0.520 Acc=0.78 ---72630.87 ms/batch
Step 00023550 L_out=0.475 Acc=0.87 ---96140.36 ms/batch
Step 00023600 L_out=0.108 Acc=0.91 ---119833.43 ms/batch
Step 00023650 L_out=0.214 Acc=0.80 ---143746.42 ms/batch
Step 00023700 L_out=0.153 Acc=0.88 ---167421.00 ms/batch
Step 00023750 L_out=0.249 Acc=0.85 ---191133.90 ms/batch
Step 00023800 L_out=0.287 Acc=0.87 ---214950.56 ms/batch
Step 00023850 L_out=0.056 Acc=0.84 ---238452.12 ms/batch
Step 00023900 L_out=0.876 Acc=0.55 ---261931.41 ms/batch
Step 00023950 L_out=0.178 Acc=0.83 ---285538.40 ms/batch
Step 00024000 L_out=0.234 Acc=0.86 ---309119.32 ms/batch
Round 3 | epoch=5, train costTime=314.87414264678955, | total_activation_sum=13607492
Round 3 | ****EPOCH 6****
Step 00024050 L_out=0.128 Acc=0.90 ---18002.51 ms/batch
Step 00024100 L_out=0.146 Acc=0.76 ---41811.96 ms/batch
Step 00024150 L_out=0.190 Acc=0.87 ---65544.53 ms/batch
Step 00024200 L_out=0.315 Acc=0.88 ---88952.50 ms/batch
Step 00024250 L_out=0.033 Acc=0.86 ---112711.02 ms/batch
Step 00024300 L_out=0.343 Acc=0.71 ---136365.19 ms/batch
Step 00024350 L_out=0.327 Acc=0.90 ---160081.88 ms/batch
Step 00024400 L_out=0.101 Acc=0.90 ---183720.39 ms/batch
Step 00024450 L_out=0.130 Acc=0.83 ---207433.76 ms/batch
Step 00024500 L_out=0.404 Acc=0.81 ---231034.18 ms/batch
Step 00024550 L_out=0.264 Acc=0.87 ---254450.35 ms/batch
Step 00024600 L_out=0.082 Acc=0.93 ---277893.77 ms/batch
Step 00024650 L_out=0.736 Acc=0.80 ---301494.79 ms/batch
Round 3 | epoch=6, train costTime=315.08136773109436, | total_activation_sum=13452617
Round 3 | ****EPOCH 7****
Step 00024700 L_out=0.435 Acc=0.76 --- 9942.81 ms/batch
Step 00024750 L_out=0.215 Acc=0.86 ---33413.95 ms/batch
Step 00024800 L_out=0.139 Acc=0.88 ---57315.85 ms/batch
Step 00024850 L_out=0.395 Acc=0.75 ---81160.69 ms/batch
Step 00024900 L_out=0.222 Acc=0.77 ---104599.66 ms/batch
Step 00024950 L_out=0.250 Acc=0.89 ---128352.25 ms/batch
Step 00025000 L_out=0.389 Acc=0.85 ---151840.81 ms/batch
Step 00025050 L_out=0.064 Acc=0.87 ---175570.93 ms/batch
Step 00025100 L_out=0.100 Acc=0.85 ---199229.55 ms/batch
Step 00025150 L_out=0.112 Acc=0.85 ---222884.22 ms/batch
Step 00025200 L_out=0.278 Acc=0.88 ---246562.15 ms/batch
Step 00025250 L_out=0.143 Acc=0.84 ---270211.47 ms/batch
Step 00025300 L_out=0.331 Acc=0.95 ---293885.99 ms/batch
Round 3 | epoch=7, train costTime=315.49935936927795, | total_activation_sum=13599659
Round 3 | ****EPOCH 8****
Step 00025350 L_out=0.255 Acc=0.85 --- 1862.48 ms/batch
Step 00025400 L_out=0.197 Acc=0.85 ---25445.97 ms/batch
Step 00025450 L_out=0.054 Acc=0.88 ---49127.98 ms/batch
Step 00025500 L_out=0.125 Acc=0.92 ---72854.68 ms/batch
Step 00025550 L_out=0.134 Acc=0.86 ---96355.11 ms/batch
Step 00025600 L_out=0.549 Acc=0.71 ---119797.17 ms/batch
Step 00025650 L_out=0.118 Acc=0.90 ---143506.32 ms/batch
Step 00025700 L_out=0.156 Acc=0.87 ---166947.17 ms/batch
Step 00025750 L_out=0.091 Acc=0.90 ---190405.24 ms/batch
Step 00025800 L_out=0.171 Acc=0.87 ---214022.60 ms/batch
Step 00025850 L_out=0.125 Acc=0.82 ---237399.03 ms/batch
Step 00025900 L_out=0.268 Acc=0.90 ---260950.64 ms/batch
Step 00025950 L_out=0.088 Acc=0.93 ---284605.95 ms/batch
Step 00026000 L_out=0.082 Acc=0.88 ---308247.24 ms/batch
Round 3 | epoch=8, train costTime=314.4586355686188, | total_activation_sum=13579495
Round 3 | ****EPOCH 9****
Step 00026050 L_out=0.211 Acc=0.92 ---17388.10 ms/batch
Step 00026100 L_out=0.137 Acc=0.90 ---41161.57 ms/batch
Step 00026150 L_out=0.187 Acc=0.92 ---64720.33 ms/batch
Step 00026200 L_out=0.300 Acc=0.71 ---88156.55 ms/batch
Step 00026250 L_out=0.212 Acc=0.85 ---111669.58 ms/batch
Step 00026300 L_out=0.258 Acc=0.88 ---135410.46 ms/batch
Step 00026350 L_out=0.300 Acc=0.85 ---158967.78 ms/batch
Step 00026400 L_out=0.081 Acc=0.90 ---182673.84 ms/batch
Step 00026450 L_out=0.106 Acc=0.76 ---206377.26 ms/batch
Step 00026500 L_out=0.312 Acc=0.85 ---230150.53 ms/batch
Step 00026550 L_out=0.114 Acc=0.88 ---253585.59 ms/batch
Step 00026600 L_out=0.141 Acc=0.91 ---277349.66 ms/batch
Step 00026650 L_out=0.164 Acc=0.90 ---301157.05 ms/batch
Round 3 | epoch=9, train costTime=315.39240312576294, | total_activation_sum=13433104
Round 3 | ****EPOCH 10****
Step 00026700 L_out=0.190 Acc=0.87 --- 9596.82 ms/batch
Step 00026750 L_out=0.251 Acc=0.88 ---33142.80 ms/batch
Step 00026800 L_out=0.126 Acc=0.88 ---56872.04 ms/batch
Step 00026850 L_out=0.241 Acc=0.86 ---80598.49 ms/batch
Step 00026900 L_out=0.233 Acc=0.85 ---104166.86 ms/batch
Step 00026950 L_out=0.248 Acc=0.91 ---127771.83 ms/batch
Step 00027000 L_out=0.119 Acc=0.89 ---151412.02 ms/batch
Step 00027050 L_out=0.182 Acc=0.86 ---175255.83 ms/batch
Step 00027100 L_out=0.182 Acc=0.85 ---198909.64 ms/batch
Step 00027150 L_out=0.496 Acc=0.80 ---222600.64 ms/batch
Step 00027200 L_out=0.397 Acc=0.82 ---245949.63 ms/batch
Step 00027250 L_out=0.358 Acc=0.94 ---269550.45 ms/batch
Step 00027300 L_out=0.036 Acc=0.95 ---293140.17 ms/batch
Round 3 | epoch=10, train costTime=315.2921767234802, | total_activation_sum=13549792
Round 3 | ****EPOCH 11****
Step 00027350 L_out=0.521 Acc=0.63 --- 1367.25 ms/batch
Step 00027400 L_out=0.462 Acc=0.89 ---24892.52 ms/batch
Step 00027450 L_out=0.165 Acc=0.89 ---48625.22 ms/batch
Step 00027500 L_out=0.129 Acc=0.90 ---72360.24 ms/batch
Step 00027550 L_out=0.151 Acc=0.92 ---95888.54 ms/batch
Step 00027600 L_out=0.147 Acc=0.88 ---119458.13 ms/batch
Step 00027650 L_out=0.768 Acc=0.86 ---143135.74 ms/batch
Step 00027700 L_out=0.288 Acc=0.88 ---166728.45 ms/batch
Step 00027750 L_out=0.208 Acc=0.82 ---190299.55 ms/batch
Step 00027800 L_out=0.188 Acc=0.93 ---213836.88 ms/batch
Step 00027850 L_out=0.406 Acc=0.76 ---237762.20 ms/batch
Step 00027900 L_out=0.173 Acc=0.91 ---261345.48 ms/batch
Step 00027950 L_out=0.332 Acc=0.83 ---285098.48 ms/batch
Step 00028000 L_out=0.276 Acc=0.88 ---308780.10 ms/batch
Round 3 | epoch=11, train costTime=315.3841426372528, | total_activation_sum=13750766
Round 3 | ****EPOCH 12****
Step 00028050 L_out=0.171 Acc=0.80 ---17115.69 ms/batch
Step 00028100 L_out=0.165 Acc=0.82 ---40966.89 ms/batch
Step 00028150 L_out=0.446 Acc=0.82 ---64531.82 ms/batch
Step 00028200 L_out=0.378 Acc=0.83 ---87972.55 ms/batch
Step 00028250 L_out=0.395 Acc=0.79 ---111741.33 ms/batch
Step 00028300 L_out=0.137 Acc=0.91 ---135744.82 ms/batch
Step 00028350 L_out=0.057 Acc=0.88 ---159446.82 ms/batch
Step 00028400 L_out=0.314 Acc=0.84 ---182677.42 ms/batch
Step 00028450 L_out=0.300 Acc=0.81 ---206504.34 ms/batch
Step 00028500 L_out=0.071 Acc=0.93 ---230219.09 ms/batch
Step 00028550 L_out=0.195 Acc=0.93 ---253891.17 ms/batch
Step 00028600 L_out=0.113 Acc=0.91 ---277741.57 ms/batch
Step 00028650 L_out=0.053 Acc=0.90 ---301449.34 ms/batch
Round 3 | epoch=12, train costTime=316.03280234336853, | total_activation_sum=13509721
Round 3 | ****EPOCH 13****
Step 00028700 L_out=0.092 Acc=0.81 --- 9032.07 ms/batch
Step 00028750 L_out=0.136 Acc=0.79 ---32371.28 ms/batch
Step 00028800 L_out=0.237 Acc=0.83 ---55991.99 ms/batch
Step 00028850 L_out=0.081 Acc=0.88 ---79792.11 ms/batch
Step 00028900 L_out=0.111 Acc=0.89 ---103432.53 ms/batch
Step 00028950 L_out=0.208 Acc=0.88 ---127186.99 ms/batch
Step 00029000 L_out=0.303 Acc=0.89 ---150726.58 ms/batch
Step 00029050 L_out=0.246 Acc=0.86 ---174245.94 ms/batch
Step 00029100 L_out=0.062 Acc=0.95 ---197855.53 ms/batch
Step 00029150 L_out=0.224 Acc=0.89 ---221289.20 ms/batch
Step 00029200 L_out=0.127 Acc=0.89 ---244902.43 ms/batch
Step 00029250 L_out=0.097 Acc=0.95 ---268705.52 ms/batch
Step 00029300 L_out=0.138 Acc=0.91 ---292426.01 ms/batch
Round 3 | epoch=13, train costTime=315.12832856178284, | total_activation_sum=13645409
Round 3 | ****EPOCH 14****
Step 00029350 L_out=0.330 Acc=0.84 ---  933.86 ms/batch
Step 00029400 L_out=0.125 Acc=0.88 ---24539.72 ms/batch
Step 00029450 L_out=0.090 Acc=0.95 ---48282.43 ms/batch
Step 00029500 L_out=0.601 Acc=0.74 ---72040.06 ms/batch
Step 00029550 L_out=0.137 Acc=0.88 ---95745.58 ms/batch
Step 00029600 L_out=0.066 Acc=0.91 ---119445.72 ms/batch
Step 00029650 L_out=0.220 Acc=0.84 ---142927.71 ms/batch
Step 00029700 L_out=0.087 Acc=0.84 ---166677.43 ms/batch
Step 00029750 L_out=0.045 Acc=0.94 ---190280.80 ms/batch
Step 00029800 L_out=0.333 Acc=0.85 ---213967.95 ms/batch
Step 00029850 L_out=0.203 Acc=0.92 ---237464.46 ms/batch
Step 00029900 L_out=0.177 Acc=0.90 ---261041.81 ms/batch
Step 00029950 L_out=0.064 Acc=0.91 ---284494.02 ms/batch
Step 00030000 L_out=0.085 Acc=0.95 ---308304.85 ms/batch
Round 3 | epoch=14, train costTime=315.4520399570465, | total_activation_sum=13482340
Round 3 | ****EPOCH 15****
Step 00030050 L_out=0.096 Acc=0.95 ---16336.80 ms/batch
Step 00030100 L_out=0.310 Acc=0.88 ---39838.52 ms/batch
Step 00030150 L_out=0.094 Acc=0.95 ---63223.65 ms/batch
Step 00030200 L_out=0.156 Acc=0.94 ---86739.57 ms/batch
Step 00030250 L_out=0.064 Acc=0.93 ---110685.55 ms/batch
Step 00030300 L_out=0.272 Acc=0.91 ---134175.99 ms/batch
Step 00030350 L_out=0.383 Acc=0.86 ---157790.25 ms/batch
Step 00030400 L_out=0.179 Acc=0.92 ---181552.30 ms/batch
Step 00030450 L_out=0.085 Acc=0.92 ---205026.28 ms/batch
Step 00030500 L_out=0.108 Acc=0.92 ---228927.61 ms/batch
Step 00030550 L_out=0.412 Acc=0.80 ---252389.03 ms/batch
Step 00030600 L_out=0.106 Acc=0.95 ---275991.76 ms/batch
Step 00030650 L_out=0.082 Acc=0.91 ---299793.50 ms/batch
Round 3 | epoch=15, train costTime=314.9128179550171, | total_activation_sum=13764237
Round 3 | ****EPOCH 16****
Step 00030700 L_out=0.163 Acc=0.86 --- 8432.30 ms/batch
Step 00030750 L_out=0.346 Acc=0.77 ---32157.43 ms/batch
Step 00030800 L_out=0.063 Acc=0.91 ---55762.51 ms/batch
Step 00030850 L_out=0.229 Acc=0.88 ---79405.84 ms/batch
Step 00030900 L_out=0.141 Acc=0.86 ---102972.62 ms/batch
Step 00030950 L_out=0.148 Acc=0.93 ---126609.89 ms/batch
Step 00031000 L_out=0.134 Acc=0.96 ---149990.93 ms/batch
Step 00031050 L_out=0.388 Acc=0.68 ---173492.43 ms/batch
Step 00031100 L_out=0.081 Acc=0.84 ---197109.88 ms/batch
Step 00031150 L_out=0.106 Acc=0.91 ---220767.94 ms/batch
Step 00031200 L_out=0.069 Acc=0.93 ---244378.69 ms/batch
Step 00031250 L_out=0.151 Acc=0.92 ---267983.56 ms/batch
Step 00031300 L_out=0.223 Acc=0.84 ---291777.38 ms/batch
Round 3 | epoch=16, train costTime=315.06628155708313, | total_activation_sum=13424907
Round 3 | ****EPOCH 17****
Step 00031350 L_out=0.151 Acc=0.89 ---  462.63 ms/batch
Step 00031400 L_out=0.305 Acc=0.90 ---24174.81 ms/batch
Step 00031450 L_out=0.126 Acc=0.95 ---47721.30 ms/batch
Step 00031500 L_out=0.203 Acc=0.88 ---71335.99 ms/batch
Step 00031550 L_out=0.062 Acc=0.84 ---94830.17 ms/batch
Step 00031600 L_out=0.164 Acc=0.94 ---118575.20 ms/batch
Step 00031650 L_out=0.075 Acc=0.90 ---142250.74 ms/batch
Step 00031700 L_out=0.046 Acc=0.94 ---166096.04 ms/batch
Step 00031750 L_out=0.100 Acc=0.85 ---189878.69 ms/batch
Step 00031800 L_out=0.055 Acc=0.94 ---213686.73 ms/batch
Step 00031850 L_out=0.085 Acc=0.93 ---237360.16 ms/batch
Step 00031900 L_out=0.061 Acc=0.88 ---261152.60 ms/batch
Step 00031950 L_out=0.237 Acc=0.89 ---284899.57 ms/batch
Step 00032000 L_out=0.108 Acc=0.89 ---308459.02 ms/batch
Round 3 | epoch=17, train costTime=315.946905374527, | total_activation_sum=13609183
Round 3 | epoch=18, current m_iou=0.6369094837274994
Round 3 | Best m_IoU is: 0.637, OA is: 0.895 | val costTime=117.03225874900818
Round 3 | ****EPOCH 18****
Step 00032050 L_out=0.123 Acc=0.88 ---16128.02 ms/batch
Step 00032100 L_out=0.303 Acc=0.80 ---39657.79 ms/batch
Step 00032150 L_out=0.063 Acc=0.93 ---63212.19 ms/batch
Step 00032200 L_out=0.051 Acc=0.95 ---86862.39 ms/batch
Step 00032250 L_out=0.101 Acc=0.87 ---110714.65 ms/batch
Step 00032300 L_out=0.019 Acc=0.91 ---134259.88 ms/batch
Step 00032350 L_out=0.080 Acc=0.92 ---157625.34 ms/batch
Step 00032400 L_out=0.169 Acc=0.96 ---181337.31 ms/batch
Step 00032450 L_out=0.131 Acc=0.93 ---205174.04 ms/batch
Step 00032500 L_out=0.021 Acc=0.95 ---228662.08 ms/batch
Step 00032550 L_out=0.176 Acc=0.88 ---252194.01 ms/batch
Step 00032600 L_out=0.184 Acc=0.82 ---275857.96 ms/batch
Step 00032650 L_out=0.119 Acc=0.93 ---299633.23 ms/batch
Round 3 | epoch=18, train costTime=315.1331913471222, | total_activation_sum=13635061
Round 3 | epoch=19, current m_iou=0.6549185653035701
Round 3 | Best m_IoU is: 0.655, OA is: 0.911 | val costTime=116.7045967578888
Round 3 | ****EPOCH 19****
Step 00032700 L_out=0.273 Acc=0.81 --- 7992.41 ms/batch
Step 00032750 L_out=0.101 Acc=0.86 ---31612.18 ms/batch
Step 00032800 L_out=0.139 Acc=0.90 ---55396.29 ms/batch
Step 00032850 L_out=0.122 Acc=0.80 ---79056.68 ms/batch
Step 00032900 L_out=0.110 Acc=0.87 ---102715.02 ms/batch
Step 00032950 L_out=0.209 Acc=0.87 ---126360.19 ms/batch
Step 00033000 L_out=0.078 Acc=0.92 ---149965.38 ms/batch
Step 00033050 L_out=0.151 Acc=0.88 ---173512.49 ms/batch
Step 00033100 L_out=0.215 Acc=0.88 ---197170.67 ms/batch
Step 00033150 L_out=0.031 Acc=0.91 ---220938.92 ms/batch
Step 00033200 L_out=0.032 Acc=0.94 ---244522.60 ms/batch
Step 00033250 L_out=0.386 Acc=0.79 ---268141.95 ms/batch
Step 00033300 L_out=0.105 Acc=0.88 ---291768.70 ms/batch
Step 00033350 L_out=0.231 Acc=0.91 ---315360.37 ms/batch
Round 3 | epoch=19, train costTime=315.3607997894287, | total_activation_sum=13638494
Round 3 | epoch=20, current m_iou=0.6611996300958869
Round 3 | Best m_IoU is: 0.661, OA is: 0.912 | val costTime=117.4326844215393
Round 3 | ****EPOCH 20****
Step 00033400 L_out=0.391 Acc=0.82 ---23559.18 ms/batch
Step 00033450 L_out=0.076 Acc=0.92 ---47114.24 ms/batch
Step 00033500 L_out=0.108 Acc=0.91 ---70713.84 ms/batch
Step 00033550 L_out=0.079 Acc=0.91 ---94347.52 ms/batch
Step 00033600 L_out=0.134 Acc=0.84 ---117992.06 ms/batch
Step 00033650 L_out=0.047 Acc=0.88 ---141514.20 ms/batch
Step 00033700 L_out=0.026 Acc=0.91 ---165261.10 ms/batch
Step 00033750 L_out=0.148 Acc=0.91 ---188723.75 ms/batch
Step 00033800 L_out=0.021 Acc=0.96 ---212161.45 ms/batch
Step 00033850 L_out=0.027 Acc=0.93 ---235747.81 ms/batch
Step 00033900 L_out=0.143 Acc=0.89 ---259504.43 ms/batch
Step 00033950 L_out=0.047 Acc=0.89 ---282978.72 ms/batch
Step 00034000 L_out=0.119 Acc=0.86 ---306448.21 ms/batch
Round 3 | epoch=20, train costTime=314.57510209083557, | total_activation_sum=13558000
Round 3 | epoch=21, current m_iou=0.6664937058279057
Round 3 | Best m_IoU is: 0.666, OA is: 0.914 | val costTime=117.04986500740051
Round 3 | ****EPOCH 21****
Step 00034050 L_out=0.129 Acc=0.96 ---15688.45 ms/batch
Step 00034100 L_out=0.147 Acc=0.95 ---39304.34 ms/batch
Step 00034150 L_out=0.134 Acc=0.93 ---62765.01 ms/batch
Step 00034200 L_out=0.060 Acc=0.92 ---86230.54 ms/batch
Step 00034250 L_out=0.131 Acc=0.88 ---109637.29 ms/batch
Step 00034300 L_out=0.099 Acc=0.91 ---133157.37 ms/batch
Step 00034350 L_out=0.116 Acc=0.85 ---156807.54 ms/batch
Step 00034400 L_out=0.102 Acc=0.87 ---180488.06 ms/batch
Step 00034450 L_out=0.039 Acc=0.84 ---204156.85 ms/batch
Step 00034500 L_out=0.047 Acc=0.97 ---227646.72 ms/batch
Step 00034550 L_out=0.102 Acc=0.96 ---251279.89 ms/batch
Step 00034600 L_out=0.142 Acc=0.94 ---274952.94 ms/batch
Step 00034650 L_out=0.032 Acc=0.94 ---298594.49 ms/batch
Round 3 | epoch=21, train costTime=314.80233550071716, | total_activation_sum=13592138
Round 3 | epoch=22, current m_iou=0.6547949976085958
Round 3 | Best m_IoU is: 0.666, OA is: 0.914 | val costTime=116.52833437919617
Round 3 | ****EPOCH 22****
Step 00034700 L_out=0.040 Acc=0.93 --- 7544.41 ms/batch
Step 00034750 L_out=0.035 Acc=0.94 ---31193.14 ms/batch
Step 00034800 L_out=0.139 Acc=0.94 ---54760.27 ms/batch
Step 00034850 L_out=0.500 Acc=0.89 ---78307.67 ms/batch
Step 00034900 L_out=0.050 Acc=0.92 ---101741.24 ms/batch
Step 00034950 L_out=0.210 Acc=0.84 ---125218.20 ms/batch
Step 00035000 L_out=0.112 Acc=0.91 ---149014.17 ms/batch
Step 00035050 L_out=0.105 Acc=0.88 ---172585.95 ms/batch
Step 00035100 L_out=0.138 Acc=0.95 ---196069.98 ms/batch
Step 00035150 L_out=0.107 Acc=0.89 ---219720.19 ms/batch
Step 00035200 L_out=0.141 Acc=0.91 ---243364.50 ms/batch
Step 00035250 L_out=0.101 Acc=0.93 ---266899.14 ms/batch
Step 00035300 L_out=0.125 Acc=0.97 ---290281.18 ms/batch
Step 00035350 L_out=0.211 Acc=0.86 ---313926.48 ms/batch
Round 3 | epoch=22, train costTime=314.3903992176056, | total_activation_sum=13614679
Round 3 | epoch=23, current m_iou=0.6714432178416965
Round 3 | Best m_IoU is: 0.671, OA is: 0.919 | val costTime=116.88722324371338
Round 3 | ****EPOCH 23****
Step 00035400 L_out=0.026 Acc=0.93 ---23284.66 ms/batch
Step 00035450 L_out=0.149 Acc=0.87 ---46659.65 ms/batch
Step 00035500 L_out=0.079 Acc=0.94 ---70399.53 ms/batch
Step 00035550 L_out=0.037 Acc=0.93 ---93885.99 ms/batch
Step 00035600 L_out=0.149 Acc=0.91 ---117525.15 ms/batch
Step 00035650 L_out=0.074 Acc=0.93 ---141219.21 ms/batch
Step 00035700 L_out=0.232 Acc=0.92 ---164673.09 ms/batch
Step 00035750 L_out=0.048 Acc=0.91 ---188289.58 ms/batch
Step 00035800 L_out=0.094 Acc=0.94 ---211797.02 ms/batch
Step 00035850 L_out=0.030 Acc=0.92 ---235560.70 ms/batch
Step 00035900 L_out=0.014 Acc=0.94 ---259180.69 ms/batch
Step 00035950 L_out=0.059 Acc=0.90 ---282821.71 ms/batch
Step 00036000 L_out=0.031 Acc=0.90 ---306412.51 ms/batch
Round 3 | epoch=23, train costTime=314.8916895389557, | total_activation_sum=13575033
Round 3 | epoch=24, current m_iou=0.672217309350398
Round 3 | Best m_IoU is: 0.672, OA is: 0.919 | val costTime=117.30643820762634
Round 3 | ****EPOCH 24****
Step 00036050 L_out=0.016 Acc=0.96 ---15083.75 ms/batch
Step 00036100 L_out=0.045 Acc=0.85 ---38729.76 ms/batch
Step 00036150 L_out=0.168 Acc=0.89 ---62394.60 ms/batch
Step 00036200 L_out=0.060 Acc=0.90 ---86036.66 ms/batch
Step 00036250 L_out=0.065 Acc=0.92 ---109931.39 ms/batch
Step 00036300 L_out=0.098 Acc=0.90 ---133555.39 ms/batch
Step 00036350 L_out=0.062 Acc=0.93 ---157159.51 ms/batch
Step 00036400 L_out=0.044 Acc=0.93 ---180962.57 ms/batch
Step 00036450 L_out=0.055 Acc=0.95 ---204713.03 ms/batch
Step 00036500 L_out=0.031 Acc=0.97 ---228361.95 ms/batch
Step 00036550 L_out=0.086 Acc=0.90 ---252226.51 ms/batch
Step 00036600 L_out=0.134 Acc=0.88 ---276306.36 ms/batch
Step 00036650 L_out=0.189 Acc=0.91 ---300235.80 ms/batch
Round 3 | epoch=24, train costTime=316.8421356678009, | total_activation_sum=13525638
Round 3 | epoch=25, current m_iou=0.6733063210336487
Round 3 | Best m_IoU is: 0.673, OA is: 0.919 | val costTime=117.3683090209961
Round 3 | ****EPOCH 25****
Step 00036700 L_out=0.026 Acc=0.90 --- 7239.94 ms/batch
Step 00036750 L_out=0.038 Acc=0.92 ---30773.08 ms/batch
Step 00036800 L_out=0.162 Acc=0.90 ---54573.42 ms/batch
Step 00036850 L_out=0.101 Acc=0.89 ---77982.55 ms/batch
Step 00036900 L_out=0.631 Acc=0.89 ---101580.50 ms/batch
Step 00036950 L_out=0.037 Acc=0.91 ---125226.15 ms/batch
Step 00037000 L_out=0.090 Acc=0.83 ---148886.86 ms/batch
Step 00037050 L_out=0.053 Acc=0.90 ---172568.22 ms/batch
Step 00037100 L_out=0.116 Acc=0.95 ---196269.60 ms/batch
Step 00037150 L_out=0.273 Acc=0.91 ---219776.01 ms/batch
Step 00037200 L_out=0.192 Acc=0.86 ---243516.54 ms/batch
Step 00037250 L_out=0.081 Acc=0.85 ---266985.13 ms/batch
Step 00037300 L_out=0.089 Acc=0.90 ---290462.11 ms/batch
Step 00037350 L_out=0.021 Acc=0.93 ---314046.34 ms/batch
Round 3 | epoch=25, train costTime=315.02187514305115, | total_activation_sum=13610403
Round 3 | epoch=26, current m_iou=0.6769475070729972
Round 3 | Best m_IoU is: 0.677, OA is: 0.918 | val costTime=116.69792032241821
Round 3 | ****EPOCH 26****
Step 00037400 L_out=0.080 Acc=0.95 ---22699.67 ms/batch
Step 00037450 L_out=0.750 Acc=0.58 ---46353.66 ms/batch
Step 00037500 L_out=0.026 Acc=0.95 ---69969.84 ms/batch
Step 00037550 L_out=0.033 Acc=0.95 ---93648.34 ms/batch
Step 00037600 L_out=0.099 Acc=0.96 ---117319.70 ms/batch
Step 00037650 L_out=0.099 Acc=0.95 ---140835.66 ms/batch
Step 00037700 L_out=0.190 Acc=0.90 ---164718.87 ms/batch
Step 00037750 L_out=0.072 Acc=0.88 ---188185.04 ms/batch
Step 00037800 L_out=0.051 Acc=0.94 ---211623.98 ms/batch
Step 00037850 L_out=0.023 Acc=0.94 ---235274.14 ms/batch
Step 00037900 L_out=0.103 Acc=0.91 ---258865.85 ms/batch
Step 00037950 L_out=0.014 Acc=0.95 ---282488.07 ms/batch
Step 00038000 L_out=0.225 Acc=0.79 ---306155.75 ms/batch
Round 3 | epoch=26, train costTime=315.17131304740906, | total_activation_sum=13594422
Round 3 | epoch=27, current m_iou=0.68011993584351
Round 3 | Best m_IoU is: 0.680, OA is: 0.919 | val costTime=116.66391134262085
Round 3 | ****EPOCH 27****
Step 00038050 L_out=0.028 Acc=0.94 ---14604.62 ms/batch
Step 00038100 L_out=0.031 Acc=0.85 ---38327.12 ms/batch
Step 00038150 L_out=0.073 Acc=0.90 ---61921.31 ms/batch
Step 00038200 L_out=0.236 Acc=0.74 ---85438.52 ms/batch
Step 00038250 L_out=0.236 Acc=0.80 ---109164.77 ms/batch
Step 00038300 L_out=0.320 Acc=0.87 ---132765.41 ms/batch
Step 00038350 L_out=0.152 Acc=0.88 ---156451.99 ms/batch
Step 00038400 L_out=0.085 Acc=0.91 ---180016.58 ms/batch
Step 00038450 L_out=0.029 Acc=0.90 ---203835.84 ms/batch
Step 00038500 L_out=0.108 Acc=0.92 ---227258.68 ms/batch
Step 00038550 L_out=0.028 Acc=0.97 ---250905.71 ms/batch
Step 00038600 L_out=0.014 Acc=0.95 ---274497.54 ms/batch
Step 00038650 L_out=0.127 Acc=0.90 ---298107.95 ms/batch
Round 3 | epoch=27, train costTime=314.9393198490143, | total_activation_sum=13497400
Round 3 | epoch=28, current m_iou=0.6799356759930444
Round 3 | Best m_IoU is: 0.680, OA is: 0.919 | val costTime=116.68383765220642
Round 3 | ****EPOCH 28****
Step 00038700 L_out=0.075 Acc=0.93 --- 6689.19 ms/batch
Step 00038750 L_out=0.105 Acc=0.94 ---30346.81 ms/batch
Step 00038800 L_out=0.106 Acc=0.91 ---53983.55 ms/batch
Step 00038850 L_out=0.074 Acc=0.91 ---77715.26 ms/batch
Step 00038900 L_out=0.054 Acc=0.93 ---101162.47 ms/batch
Step 00038950 L_out=0.057 Acc=0.96 ---124662.87 ms/batch
Step 00039000 L_out=0.140 Acc=0.94 ---148306.47 ms/batch
Step 00039050 L_out=0.052 Acc=0.96 ---171939.62 ms/batch
Step 00039100 L_out=0.126 Acc=0.92 ---195664.12 ms/batch
Step 00039150 L_out=0.044 Acc=0.95 ---219345.06 ms/batch
Step 00039200 L_out=0.131 Acc=0.95 ---243378.87 ms/batch
Step 00039250 L_out=0.166 Acc=0.87 ---266902.13 ms/batch
Step 00039300 L_out=0.216 Acc=0.79 ---290576.10 ms/batch
Step 00039350 L_out=0.052 Acc=0.92 ---314298.96 ms/batch
Round 3 | epoch=28, train costTime=315.7127585411072, | total_activation_sum=13703714
Round 3 | epoch=29, current m_iou=0.6758287344754892
Round 3 | Best m_IoU is: 0.680, OA is: 0.919 | val costTime=116.31165051460266
Round 3 | ****EPOCH 29****
Step 00039400 L_out=0.059 Acc=0.89 ---22116.50 ms/batch
Step 00039450 L_out=0.088 Acc=0.93 ---45703.70 ms/batch
Step 00039500 L_out=0.253 Acc=0.79 ---69236.29 ms/batch
Step 00039550 L_out=0.049 Acc=0.91 ---92840.83 ms/batch
Step 00039600 L_out=0.005 Acc=0.93 ---116582.87 ms/batch
Step 00039650 L_out=0.129 Acc=0.92 ---140302.33 ms/batch
Step 00039700 L_out=0.035 Acc=0.94 ---163757.12 ms/batch
Step 00039750 L_out=0.131 Acc=0.86 ---187519.20 ms/batch
Step 00039800 L_out=0.050 Acc=0.95 ---211136.33 ms/batch
Step 00039850 L_out=0.079 Acc=0.85 ---234753.48 ms/batch
Step 00039900 L_out=0.032 Acc=0.93 ---258451.75 ms/batch
Step 00039950 L_out=0.052 Acc=0.93 ---282138.75 ms/batch
Step 00040000 L_out=0.170 Acc=0.89 ---305844.48 ms/batch
Round 3 | epoch=29, train costTime=315.36403346061707, | total_activation_sum=13548239
Round 3 | epoch=30, current m_iou=0.6724658867993091
Round 3 | Best m_IoU is: 0.680, OA is: 0.919 | val costTime=115.78374481201172
Round 3 | ****EPOCH 30****
Round 4 | ****EPOCH 0****
Step 00040050 L_out=0.295 Acc=0.83 ---14080.87 ms/batch
Step 00040100 L_out=0.529 Acc=0.74 ---37660.94 ms/batch
Step 00040150 L_out=0.359 Acc=0.78 ---61403.07 ms/batch
Step 00040200 L_out=0.446 Acc=0.88 ---85168.13 ms/batch
Step 00040250 L_out=0.480 Acc=0.91 ---108750.36 ms/batch
Step 00040300 L_out=0.862 Acc=0.85 ---132686.40 ms/batch
Step 00040350 L_out=0.703 Acc=0.77 ---156424.90 ms/batch
Step 00040400 L_out=0.177 Acc=0.87 ---180151.99 ms/batch
Step 00040450 L_out=0.346 Acc=0.84 ---203853.80 ms/batch
Step 00040500 L_out=0.312 Acc=0.86 ---227297.56 ms/batch
Step 00040550 L_out=0.616 Acc=0.85 ---250985.94 ms/batch
Step 00040600 L_out=0.510 Acc=0.86 ---274513.95 ms/batch
Step 00040650 L_out=0.201 Acc=0.94 ---297998.42 ms/batch
Round 4 | epoch=0, train costTime=315.44512033462524, | total_activation_sum=18770953
Round 4 | ****EPOCH 1****
Step 00040700 L_out=0.214 Acc=0.90 --- 6176.68 ms/batch
Step 00040750 L_out=0.301 Acc=0.87 ---29708.84 ms/batch
Step 00040800 L_out=0.211 Acc=0.94 ---53251.87 ms/batch
Step 00040850 L_out=0.868 Acc=0.84 ---76891.77 ms/batch
Step 00040900 L_out=0.680 Acc=0.86 ---100452.15 ms/batch
Step 00040950 L_out=0.435 Acc=0.87 ---124190.44 ms/batch
Step 00041000 L_out=0.338 Acc=0.83 ---147803.92 ms/batch
Step 00041050 L_out=0.861 Acc=0.76 ---171424.38 ms/batch
Step 00041100 L_out=0.550 Acc=0.72 ---195032.78 ms/batch
Step 00041150 L_out=0.404 Acc=0.81 ---218765.99 ms/batch
Step 00041200 L_out=1.411 Acc=0.76 ---242241.62 ms/batch
Step 00041250 L_out=0.444 Acc=0.89 ---265764.44 ms/batch
Step 00041300 L_out=0.242 Acc=0.89 ---289363.54 ms/batch
Step 00041350 L_out=0.560 Acc=0.90 ---312881.71 ms/batch
Round 4 | epoch=1, train costTime=314.8566083908081, | total_activation_sum=17575000
Round 4 | ****EPOCH 2****
Step 00041400 L_out=0.819 Acc=0.79 ---21808.77 ms/batch
Step 00041450 L_out=0.289 Acc=0.86 ---45636.21 ms/batch
Step 00041500 L_out=0.733 Acc=0.83 ---69194.11 ms/batch
Step 00041550 L_out=0.246 Acc=0.92 ---92805.86 ms/batch
Step 00041600 L_out=0.293 Acc=0.86 ---116304.11 ms/batch
Step 00041650 L_out=0.206 Acc=0.94 ---139702.89 ms/batch
Step 00041700 L_out=0.633 Acc=0.88 ---163245.35 ms/batch
Step 00041750 L_out=0.575 Acc=0.77 ---186904.46 ms/batch
Step 00041800 L_out=0.383 Acc=0.88 ---210652.35 ms/batch
Step 00041850 L_out=0.265 Acc=0.87 ---234202.67 ms/batch
Step 00041900 L_out=0.373 Acc=0.94 ---257940.45 ms/batch
Step 00041950 L_out=0.269 Acc=0.87 ---281899.46 ms/batch
Step 00042000 L_out=0.241 Acc=0.93 ---305482.35 ms/batch
Round 4 | epoch=2, train costTime=315.37711000442505, | total_activation_sum=17457212
Round 4 | ****EPOCH 3****
Step 00042050 L_out=0.320 Acc=0.90 ---13659.54 ms/batch
Step 00042100 L_out=0.394 Acc=0.93 ---37284.19 ms/batch
Step 00042150 L_out=0.484 Acc=0.87 ---60962.68 ms/batch
Step 00042200 L_out=0.428 Acc=0.82 ---84435.34 ms/batch
Step 00042250 L_out=0.349 Acc=0.91 ---108158.32 ms/batch
Step 00042300 L_out=0.221 Acc=0.93 ---131521.50 ms/batch
Step 00042350 L_out=0.419 Acc=0.91 ---155244.16 ms/batch
Step 00042400 L_out=0.304 Acc=0.86 ---178878.57 ms/batch
Step 00042450 L_out=0.258 Acc=0.88 ---202536.70 ms/batch
Step 00042500 L_out=0.351 Acc=0.86 ---225979.13 ms/batch
Step 00042550 L_out=0.368 Acc=0.88 ---249550.61 ms/batch
Step 00042600 L_out=0.149 Acc=0.91 ---273147.18 ms/batch
Step 00042650 L_out=0.362 Acc=0.76 ---296638.04 ms/batch
Round 4 | epoch=3, train costTime=314.56307768821716, | total_activation_sum=17563354
Round 4 | ****EPOCH 4****
Step 00042700 L_out=0.467 Acc=0.85 --- 5605.43 ms/batch
Step 00042750 L_out=0.191 Acc=0.94 ---29187.18 ms/batch
Step 00042800 L_out=0.146 Acc=0.90 ---52824.93 ms/batch
Step 00042850 L_out=0.318 Acc=0.90 ---76433.24 ms/batch
Step 00042900 L_out=0.128 Acc=0.94 ---100086.58 ms/batch
Step 00042950 L_out=0.299 Acc=0.95 ---123811.64 ms/batch
Step 00043000 L_out=0.300 Acc=0.93 ---147625.35 ms/batch
Step 00043050 L_out=0.230 Acc=0.91 ---171354.04 ms/batch
Step 00043100 L_out=0.344 Acc=0.89 ---195019.74 ms/batch
Step 00043150 L_out=0.346 Acc=0.89 ---218625.52 ms/batch
Step 00043200 L_out=0.112 Acc=0.95 ---242412.38 ms/batch
Step 00043250 L_out=0.462 Acc=0.89 ---266200.30 ms/batch
Step 00043300 L_out=0.465 Acc=0.87 ---290004.99 ms/batch
Step 00043350 L_out=0.685 Acc=0.89 ---313565.29 ms/batch
Round 4 | epoch=4, train costTime=315.97483348846436, | total_activation_sum=17632715
Round 4 | ****EPOCH 5****
Step 00043400 L_out=0.121 Acc=0.91 ---21230.53 ms/batch
Step 00043450 L_out=0.821 Acc=0.82 ---44822.35 ms/batch
Step 00043500 L_out=0.350 Acc=0.92 ---68594.60 ms/batch
Step 00043550 L_out=0.144 Acc=0.95 ---92448.17 ms/batch
Step 00043600 L_out=0.112 Acc=0.95 ---115938.24 ms/batch
Step 00043650 L_out=0.325 Acc=0.87 ---139577.71 ms/batch
Step 00043700 L_out=0.338 Acc=0.90 ---163458.95 ms/batch
Step 00043750 L_out=0.166 Acc=0.92 ---186934.87 ms/batch
Step 00043800 L_out=0.520 Acc=0.89 ---210775.43 ms/batch
Step 00043850 L_out=0.445 Acc=0.89 ---234486.66 ms/batch
Step 00043900 L_out=0.392 Acc=0.86 ---258127.89 ms/batch
Step 00043950 L_out=0.559 Acc=0.72 ---281752.15 ms/batch
Step 00044000 L_out=0.131 Acc=0.95 ---305252.86 ms/batch
Round 4 | epoch=5, train costTime=315.6079857349396, | total_activation_sum=17560880
Round 4 | ****EPOCH 6****
Step 00044050 L_out=0.212 Acc=0.86 ---13088.40 ms/batch
Step 00044100 L_out=0.297 Acc=0.90 ---36830.92 ms/batch
Step 00044150 L_out=0.308 Acc=0.85 ---60432.75 ms/batch
Step 00044200 L_out=0.292 Acc=0.93 ---83888.03 ms/batch
Step 00044250 L_out=0.465 Acc=0.86 ---107483.33 ms/batch
Step 00044300 L_out=0.751 Acc=0.90 ---131017.30 ms/batch
Step 00044350 L_out=0.172 Acc=0.92 ---154809.90 ms/batch
Step 00044400 L_out=0.261 Acc=0.93 ---178525.59 ms/batch
Step 00044450 L_out=0.240 Acc=0.95 ---202212.47 ms/batch
Step 00044500 L_out=0.294 Acc=0.90 ---225714.72 ms/batch
Step 00044550 L_out=0.479 Acc=0.68 ---249373.76 ms/batch
Step 00044600 L_out=0.448 Acc=0.77 ---272866.48 ms/batch
Step 00044650 L_out=0.316 Acc=0.91 ---296355.65 ms/batch
Round 4 | epoch=6, train costTime=314.69947266578674, | total_activation_sum=17559642
Round 4 | ****EPOCH 7****
Step 00044700 L_out=0.576 Acc=0.86 --- 5224.61 ms/batch
Step 00044750 L_out=0.407 Acc=0.92 ---28794.39 ms/batch
Step 00044800 L_out=0.070 Acc=0.96 ---52397.49 ms/batch
Step 00044850 L_out=0.173 Acc=0.90 ---75913.20 ms/batch
Step 00044900 L_out=0.173 Acc=0.94 ---99416.52 ms/batch
Step 00044950 L_out=0.215 Acc=0.92 ---123000.50 ms/batch
Step 00045000 L_out=0.557 Acc=0.83 ---146804.58 ms/batch
Step 00045050 L_out=0.169 Acc=0.91 ---170437.02 ms/batch
Step 00045100 L_out=0.125 Acc=0.94 ---194061.53 ms/batch
Step 00045150 L_out=0.118 Acc=0.94 ---217604.29 ms/batch
Step 00045200 L_out=0.348 Acc=0.91 ---241317.05 ms/batch
Step 00045250 L_out=0.113 Acc=0.94 ---264993.18 ms/batch
Step 00045300 L_out=0.151 Acc=0.93 ---288623.06 ms/batch
Step 00045350 L_out=0.114 Acc=0.93 ---312334.95 ms/batch
Round 4 | epoch=7, train costTime=315.1405098438263, | total_activation_sum=17476803
Round 4 | ****EPOCH 8****
Step 00045400 L_out=0.402 Acc=0.92 ---20754.63 ms/batch
Step 00045450 L_out=0.143 Acc=0.93 ---44583.72 ms/batch
Step 00045500 L_out=0.193 Acc=0.86 ---68099.48 ms/batch
Step 00045550 L_out=0.123 Acc=0.89 ---91703.60 ms/batch
Step 00045600 L_out=0.234 Acc=0.92 ---115483.85 ms/batch
Step 00045650 L_out=0.184 Acc=0.91 ---139008.62 ms/batch
Step 00045700 L_out=0.194 Acc=0.95 ---162756.90 ms/batch
Step 00045750 L_out=0.083 Acc=0.94 ---186366.40 ms/batch
Step 00045800 L_out=0.665 Acc=0.85 ---210079.95 ms/batch
Step 00045850 L_out=0.131 Acc=0.94 ---233559.04 ms/batch
Step 00045900 L_out=0.146 Acc=0.94 ---257186.51 ms/batch
Step 00045950 L_out=0.272 Acc=0.84 ---280896.86 ms/batch
Step 00046000 L_out=0.101 Acc=0.95 ---304605.10 ms/batch
Round 4 | epoch=8, train costTime=315.4054388999939, | total_activation_sum=17594793
Round 4 | ****EPOCH 9****
Step 00046050 L_out=0.170 Acc=0.89 ---12633.20 ms/batch
Step 00046100 L_out=0.115 Acc=0.86 ---36246.79 ms/batch
Step 00046150 L_out=0.123 Acc=0.84 ---59849.02 ms/batch
Step 00046200 L_out=0.255 Acc=0.91 ---83692.86 ms/batch
Step 00046250 L_out=0.114 Acc=0.95 ---107048.28 ms/batch
Step 00046300 L_out=0.618 Acc=0.68 ---130575.10 ms/batch
Step 00046350 L_out=0.276 Acc=0.86 ---154203.13 ms/batch
Step 00046400 L_out=0.233 Acc=0.93 ---177751.58 ms/batch
Step 00046450 L_out=0.268 Acc=0.95 ---201271.92 ms/batch
Step 00046500 L_out=0.168 Acc=0.86 ---224977.56 ms/batch
Step 00046550 L_out=0.217 Acc=0.92 ---248710.96 ms/batch
Step 00046600 L_out=0.085 Acc=0.94 ---272322.27 ms/batch
Step 00046650 L_out=0.243 Acc=0.88 ---295867.02 ms/batch
Round 4 | epoch=9, train costTime=314.7024555206299, | total_activation_sum=17601789
Round 4 | ****EPOCH 10****
Step 00046700 L_out=0.236 Acc=0.91 --- 4651.33 ms/batch
Step 00046750 L_out=0.585 Acc=0.83 ---28286.23 ms/batch
Step 00046800 L_out=0.223 Acc=0.93 ---51901.14 ms/batch
Step 00046850 L_out=0.140 Acc=0.93 ---75343.61 ms/batch
Step 00046900 L_out=0.196 Acc=0.93 ---98803.53 ms/batch
Step 00046950 L_out=0.163 Acc=0.92 ---122502.64 ms/batch
Step 00047000 L_out=0.206 Acc=0.90 ---146011.37 ms/batch
Step 00047050 L_out=0.142 Acc=0.95 ---169800.41 ms/batch
Step 00047100 L_out=0.071 Acc=0.95 ---193411.14 ms/batch
Step 00047150 L_out=0.199 Acc=0.94 ---217103.60 ms/batch
Step 00047200 L_out=0.156 Acc=0.93 ---240767.64 ms/batch
Step 00047250 L_out=0.141 Acc=0.90 ---264344.64 ms/batch
Step 00047300 L_out=0.207 Acc=0.91 ---287992.32 ms/batch
Step 00047350 L_out=0.510 Acc=0.91 ---311482.71 ms/batch
Round 4 | epoch=10, train costTime=314.8135676383972, | total_activation_sum=17516332
Round 4 | ****EPOCH 11****
Step 00047400 L_out=0.213 Acc=0.94 ---20274.22 ms/batch
Step 00047450 L_out=0.132 Acc=0.86 ---43955.86 ms/batch
Step 00047500 L_out=0.571 Acc=0.76 ---67640.76 ms/batch
Step 00047550 L_out=0.070 Acc=0.96 ---91355.50 ms/batch
Step 00047600 L_out=0.049 Acc=0.94 ---115051.48 ms/batch
Step 00047650 L_out=0.284 Acc=0.83 ---138626.65 ms/batch
Step 00047700 L_out=0.221 Acc=0.92 ---162221.11 ms/batch
Step 00047750 L_out=0.209 Acc=0.94 ---185606.52 ms/batch
Step 00047800 L_out=0.112 Acc=0.91 ---209318.83 ms/batch
Step 00047850 L_out=0.074 Acc=0.96 ---232826.45 ms/batch
Step 00047900 L_out=0.143 Acc=0.81 ---256408.01 ms/batch
Step 00047950 L_out=0.136 Acc=0.94 ---279928.12 ms/batch
Step 00048000 L_out=0.300 Acc=0.89 ---303325.63 ms/batch
Round 4 | epoch=11, train costTime=314.6585621833801, | total_activation_sum=17518261
Round 4 | ****EPOCH 12****
Step 00048050 L_out=0.332 Acc=0.85 ---12215.32 ms/batch
Step 00048100 L_out=0.150 Acc=0.93 ---35671.72 ms/batch
Step 00048150 L_out=0.154 Acc=0.93 ---59357.75 ms/batch
Step 00048200 L_out=0.375 Acc=0.93 ---83203.04 ms/batch
Step 00048250 L_out=0.086 Acc=0.94 ---106835.71 ms/batch
Step 00048300 L_out=0.175 Acc=0.93 ---130332.43 ms/batch
Step 00048350 L_out=0.161 Acc=0.94 ---153943.14 ms/batch
Step 00048400 L_out=0.496 Acc=0.91 ---177406.23 ms/batch
Step 00048450 L_out=0.065 Acc=0.95 ---200939.82 ms/batch
Step 00048500 L_out=0.190 Acc=0.92 ---224591.88 ms/batch
Step 00048550 L_out=0.203 Acc=0.92 ---248186.00 ms/batch
Step 00048600 L_out=0.391 Acc=0.92 ---271700.17 ms/batch
Step 00048650 L_out=0.294 Acc=0.90 ---295348.92 ms/batch
Round 4 | epoch=12, train costTime=314.73128056526184, | total_activation_sum=17720499
Round 4 | ****EPOCH 13****
Step 00048700 L_out=0.213 Acc=0.92 --- 4169.85 ms/batch
Step 00048750 L_out=0.153 Acc=0.94 ---27921.05 ms/batch
Step 00048800 L_out=0.222 Acc=0.93 ---51467.76 ms/batch
Step 00048850 L_out=0.258 Acc=0.94 ---74927.60 ms/batch
Step 00048900 L_out=0.099 Acc=0.94 ---98663.24 ms/batch
Step 00048950 L_out=0.183 Acc=0.90 ---122004.68 ms/batch
Step 00049000 L_out=0.146 Acc=0.94 ---145466.89 ms/batch
Step 00049050 L_out=0.119 Acc=0.94 ---168840.41 ms/batch
Step 00049100 L_out=0.224 Acc=0.92 ---192479.20 ms/batch
Step 00049150 L_out=0.153 Acc=0.90 ---216100.16 ms/batch
Step 00049200 L_out=0.175 Acc=0.92 ---239682.71 ms/batch
Step 00049250 L_out=0.335 Acc=0.86 ---263390.89 ms/batch
Step 00049300 L_out=0.320 Acc=0.93 ---287097.66 ms/batch
Step 00049350 L_out=0.225 Acc=0.88 ---310942.63 ms/batch
Round 4 | epoch=13, train costTime=314.6880280971527, | total_activation_sum=17555805
Round 4 | ****EPOCH 14****
Step 00049400 L_out=0.043 Acc=0.96 ---19832.53 ms/batch
Step 00049450 L_out=0.112 Acc=0.93 ---43488.87 ms/batch
Step 00049500 L_out=0.147 Acc=0.87 ---67093.14 ms/batch
Step 00049550 L_out=0.156 Acc=0.92 ---90723.84 ms/batch
Step 00049600 L_out=0.154 Acc=0.94 ---114209.78 ms/batch
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.259 Acc=0.15 ---25715.22 ms/batch
Step 00000100 L_out=0.112 Acc=0.33 ---50011.15 ms/batch
Step 00000150 L_out=0.068 Acc=0.55 ---74262.54 ms/batch
Step 00000200 L_out=0.221 Acc=0.70 ---98621.15 ms/batch
Step 00000250 L_out=1.506 Acc=0.35 ---122528.67 ms/batch
Step 00000300 L_out=0.185 Acc=0.38 ---148106.48 ms/batch
Step 00000350 L_out=0.124 Acc=0.26 ---174681.30 ms/batch
Step 00000400 L_out=0.177 Acc=0.36 ---201906.71 ms/batch
Step 00000450 L_out=0.036 Acc=0.46 ---228779.77 ms/batch
Step 00000500 L_out=0.195 Acc=0.68 ---255327.84 ms/batch
Step 00000550 L_out=0.735 Acc=0.29 ---281700.30 ms/batch
Step 00000600 L_out=0.203 Acc=0.59 ---308468.75 ms/batch
Step 00000650 L_out=0.021 Acc=0.72 ---335521.23 ms/batch
Round 2 | epoch=0, train costTime=344.9700939655304, | total_activation_sum=3902380
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.260 Acc=0.27 ---17790.00 ms/batch
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.704 Acc=0.39 ---26394.30 ms/batch
Step 00000100 L_out=0.088 Acc=0.05 ---50509.57 ms/batch
Step 00000150 L_out=0.079 Acc=0.23 ---74256.82 ms/batch
Step 00000200 L_out=0.092 Acc=0.08 ---98342.48 ms/batch
Step 00000250 L_out=0.090 Acc=0.38 ---122912.96 ms/batch
Step 00000300 L_out=0.499 Acc=0.35 ---146735.62 ms/batch
Step 00000350 L_out=0.071 Acc=0.33 ---171490.64 ms/batch
Step 00000400 L_out=0.091 Acc=0.29 ---195690.76 ms/batch
Step 00000450 L_out=0.106 Acc=0.07 ---219234.04 ms/batch
Step 00000500 L_out=0.225 Acc=0.53 ---243367.87 ms/batch
Step 00000550 L_out=0.039 Acc=0.44 ---267604.99 ms/batch
Step 00000600 L_out=0.380 Acc=0.42 ---291905.99 ms/batch
Step 00000650 L_out=0.103 Acc=0.48 ---315999.85 ms/batch
Round 2 | epoch=0, train costTime=324.214736700058, | total_activation_sum=3936806
Round 2 | ****EPOCH 1****
Step 00000700 L_out=0.255 Acc=0.42 ---15967.15 ms/batch
Step 00000750 L_out=0.711 Acc=0.62 ---39814.70 ms/batch
Step 00000800 L_out=0.528 Acc=0.34 ---63795.83 ms/batch
Step 00000850 L_out=0.255 Acc=0.48 ---87274.55 ms/batch
Step 00000900 L_out=0.183 Acc=0.40 ---111449.73 ms/batch
Step 00000950 L_out=0.308 Acc=0.41 ---135406.97 ms/batch
Step 00001000 L_out=0.489 Acc=0.61 ---159262.99 ms/batch
Round 2 | ****EPOCH 0****
Step 00000050 L_out=0.034 Acc=0.33 ---26162.39 ms/batch
Step 00000100 L_out=0.242 Acc=0.40 ---50073.31 ms/batch
Step 00000150 L_out=0.161 Acc=0.38 ---74297.72 ms/batch
