Initial Commit

.gitignore

+# Compiled source #
+###################
+
+*.com
+*.class
+*.dll
+*.exe
+*.o
+*.so
+*__pycache__
+
+# Packages #
+############
+
+# it's better to unpack these files and commit the raw source
+# git has its own built in compression methods
+*.7z
+*.dmg
+*.gz
+*.iso
+*.jar
+*.rar
+*.tar
+*.zip
+
+
+# Logs and databases #
+######################
+*.log
+*.sql
+*.sqlite
+
+
+# OS generated files #
+######################
+.DS_Store
+.DS_Store?
+._*
+.Spotlight-V100
+.Trashes
+ehthumbs.db
+Thumbs.db
+*.swp
+tags
+*.pem
+
+# project specific files #
+##########################
+emulation-exp/code/tmp/
+# emulation-exp/code/kex/data
+# emulation-exp/code/sig/data

README.md

+# PQC in TLS
+
+Mit Hilfe dieses Projekts sollen verschiedene PQC Algorithmen für Schlüsselaustausch und Authentifizierung auf ihre Performanz hin evaluiert werden. Dazu werden die Algorithmen in TLS 1.3  integriert und anhand eines Frameworks des Linux Kernels zur Netzwerkemulation getestet.
+
+## Wichtige Komponenten
+
+### Open Quantum Save
+Das [Open Quantum Save (OQS)](https://github.com/open-quantum-safe)  ist ein open source Projekt, welches die Entwicklung und Intergration von PQC Techniken unterstützen soll. Es beinhaltet unter anderem verschiedene Implementierungen für die PQC Kandidaten des [NIST](https://www.nist.gov) [Standardisierungsprozesses](https://csrc.nist.gov/projects/post-quantum-cryptography).
+
+#### OpenSLL
+[OQS-OpenSSL](https://github.com/open-quantum-safe/openssl) ist im OQS Projekt verankert und bietet eine open source Bibliothek für die Verwendung von TLS 1.3 mit PQC Algorithmen. 
+
+### PQ TLS Benchmark Framework
+Dieses [Benchmarking Framework](https://github.com/xvzcf/pq-tls-benchmark) wurde in einem [Paper von Christian Paquin, Douglas Stebila, and Goutam Tamvada](https://eprint.iacr.org/2019/1447) in 2019 beschrieben und dient der Evaluierung von PQC Algorithmen in TLS 1.3. Für eine realistische Netzwerk Emulation wird hierzu das Linux Kernel Tool [netem](https://www.linux.org/docs/man8/tc-netem.html) genutzt, um somit Phänomene wie Paketverlust zu simulieren. Zum Vergleich wurden zudem Experimente in realer Umgung durchgeführt. Hierbei wurden Server in den USA, EU, und Australien genutzt, um TLS 1.3 Verbindungen mittels PQC Algorithmen aufzubauen. das Framework ist als Open Source Projekt frei verfpgbar und soll in diesem Projekt genutzt und weiter entwickelt werden, um weitere PQC Algorithmen und Konfigurationen zu evaluieren.
+
+## Anforderungen an die Testumgebung
+* Ubuntu (18.04+)
+* Linux Kernel (4.12+)
+* Für die Nutzung der PQC Algorithmen innerhalb von liboqs ist die CPU-Unterstützung folgender Erweiterungen notwendig
+    > avx2
+    > bmi1
+    > bmi2
+    > popcnt
+    > sse2
+
+## Installation
+Sofern die oben beschrieben Bedingungen erfüllt sind, kann die Installation aller notwendigen Komponenten anhand des Skripts [install-prereqs-ubuntu.sh](https://code.fbi.h-da.de/aw/prj/athenepqc/pqc-in-tls/-/blob/all_algorithms2/pq-tls-benchmark-framework/emulation-exp/code/install-prereqs-ubuntu.sh) erfolgen.
+
+## Experimente
+Für die Experimente muss der Ordner 'pq-tls-benchmark-framework/emulation-exp/code/kex' aufgesucht werden. Anschließend kann das Skript [run.sh](https://code.fbi.h-da.de/aw/prj/athenepqc/pqc-in-tls/-/blob/all_algorithms2/pq-tls-benchmark-framework/emulation-exp/code/kex/scripts/run.sh) aufgerufen werden.

pq-tls-benchmark-framework/emulation-exp/README.md

+## Requirements
+
+1. A sufficiently recent version of Ubuntu (18.04+)
+2. A sufficiently recent version of the Linux kernel (4.12+)
+3. If you want to use the liboqs version used by the most recent version of the paper, a CPU that supports the following extensions:
+	- avx2
+	- bmi1
+	- bmi2
+	- popcnt
+	- sse2
+
+## Steps
+
+1. Run `./install-prereqs-ubuntu.sh`.
+
+	- If your CPU does not satisfy requirement 3 above, you might have to either modify the build files in the [xvzcf/pq-tls-experiment branch](https://github.com/xvzcf/liboqs) yourself or change [this line](https://github.com/xvzcf/liboqs/blob/pq-tls-experiment/config/detect-cpu-extensions.c) to pull the most current liboqs from `https://github.com/open-quantum-safe/liboqs`
+
+
+2. To run the key-exchange experiments, navigate to `kex`, and run `./setup.sh` followed by `python3 experiment.py`. To cleanup, run `./teardown.sh`.
+
+3. To run the signature experiments, navigate to `sig`, and run `./setup.sh` followed by `./run.sh`. To cleanup, run `./teardown.sh`.
+
+Make sure to cleanup before switching from the key-exchange experiments to the signature ones and vice-versa.

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1/kyber512.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0,13.266849385,0.7558349889100774,13.4162345,13.82802675,14.214887049999998
+0,13.567407094999998,0.5607868888240042,13.509291000000001,13.9988,14.4373627
+0,31.963581095,126.6735683634942,13.370038000000001,13.753176499999999,14.312366399999998
+0,54.39144776999999,190.13148186259903,13.809504,14.006135749999999,218.355083
+0,37.078603125,144.504229937992,13.6246935,13.8816725,14.420904249999998
+0,24.971605430000004,80.14884410064103,13.410872999999999,13.73490675,15.278455999999714
+0,92.69593775499999,297.61886439379066,13.85478,13.97314375,1015.5982911499999
+0,131.37505541000002,413.1256406518274,13.868258,14.07405975,1029.0671423000001
+0,103.55377402999999,284.82299414317686,13.869529,14.106074999999999,1027.0333798
+0,173.73109462,432.8399870374709,13.9340875,14.455715,1030.36302725
+0,174.72601998000002,422.4646555611461,13.931403,32.251432,1035.5811964499999
+0,188.42779034500003,497.01505795887573,13.8580185,14.116303499999999,1043.17663795
+0,238.18722149,470.7120617774627,13.901928,227.39005450000002,1045.0452298500002
+0,234.42211835499998,455.3528636127446,13.9428615,226.46859325,1041.90576905
+0,363.838816995,785.2795944145624,13.957184,229.9104755,1458.8890016499993
+0,430.66652610000006,906.1113155634877,14.029122999999998,244.95004,1446.8479294999984
+0,376.67729810000003,885.3835896423722,14.0179455,234.203404,1269.184149749996
+0,575.073092565,1357.4775322132189,14.070137500000001,1025.677399,2630.0910800999923
+0,457.10676331499997,870.9579474538086,14.037961,1021.27153575,2070.307744299993
+0,432.45875127500005,850.5068816263612,14.066242,1016.2643265,1276.2521829999969
+0,514.4951957850001,1080.5041142604541,14.0872235,1018.44863825,3032.7470725499993
+0,567.17725201,1078.5650175472115,23.315381,1027.69325375,3034.7191387499997
+0,798.066415015,1343.346000139378,226.5905335,1034.15650525,3402.0044153999993
+0,1044.2827663899998,2265.69302874243,226.3650675,1091.3683805,4056.6446810999996

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1/kyber90s512.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0,13.262827345,0.7387081194925001,13.4586345,13.8282355,14.02089895
+0,19.523436585000002,73.96298824889132,13.517473500000001,13.83863075,14.0920631
+0,23.791283540000006,101.43640344833007,13.3824545,13.788216,14.46698075
+0,35.90384561,143.4679647160682,13.5285375,13.82413,14.260292099999992
+0,54.64921159000001,285.34705675604243,13.585364,13.83182525,14.435392550000001
+0,30.96327002,107.41013000840384,13.4956575,13.77686325,33.14235344999991
+0,58.293392725000004,196.05085215326974,13.767045,14.0355095,229.41426095
+0,139.4242448,386.88940143650825,13.783667,13.94817375,1036.9730321499999
+0,148.31549671000002,366.08268137086634,13.741389999999999,13.953471999999998,1031.9150994
+0,211.31141141,494.9851346058419,13.8132175,14.038765499999998,1045.7256459
+0,140.91889290999998,323.38086897808193,13.817998,14.002022750000002,1030.23569005
+0,245.51416684,533.6645660214292,13.810157499999999,33.287223,1044.6684148
+0,189.44893489999998,453.3129846569311,13.866458999999999,33.796769,1042.02739815
+0,251.49981485000004,780.1776587918598,13.820987500000001,34.468268,1065.936227499997
+0,402.81202694,919.9124994926012,13.880354,292.15927525,1255.2245462499998
+0,382.87903408,700.1030069667082,13.9221835,499.11614199999997,1246.5085121
+0,389.15914665500003,880.3540956079506,13.9478515,247.97119325,1251.2394709999999
+0,494.712966855,1369.9592115269259,13.925633,250.82354974999998,1820.253299749996
+0,430.91739381,1029.0660365238728,13.9942895,247.20282500000002,2003.325724049999
+0,656.96952907,1342.1229158330584,32.624171000000004,1028.3799625,3084.8551664999973
+0,793.41232674,2022.2513997544283,22.6323525,1025.71378275,3045.2342037999997
+0,694.30068729,1365.7614698531365,32.308882499999996,1034.4417624999999,3045.3904218999996
+0,900.4572355150001,3260.529658700791,227.84733599999998,1029.139168,3061.1596129499994
+0,929.9314042,1732.4477135772097,39.3771615,1042.8053015,3641.5065827999997

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1/lightsaber.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0,13.893587754999999,0.4468426611168353,13.948907,14.185699,14.58797575
+0,20.524566999999998,74.46614458755182,13.435533,13.79351175,14.098491999999998
+0,31.679878640000002,125.6006094818216,13.431738,13.84529825,14.11549465
+0,23.484084855000003,99.79572354510677,13.383496000000001,13.99479975,14.4315237
+0,53.20144069999999,176.34541162539742,13.7195815,13.906614999999999,220.55641874999995
+0,79.715656465,290.97570221292324,13.856295,14.00298475,224.70488585
+0,87.64132776999999,333.8061701232193,13.716184,13.96442175,225.3919363
+0,98.80134878500002,311.51982279983935,13.8690485,13.969331,1015.3839588999999
+0,116.27922448999998,285.29324268907567,13.8972005,14.09061975,1020.8340234
+0,206.85978596500001,466.5091541720702,13.931792999999999,222.55730100000002,1037.05938605
+0,209.94293978000002,460.08272924460533,13.935687000000001,65.54913475000001,1038.2301078
+0,284.776372915,632.8923613700741,13.9217785,226.87626925,1044.553358
+0,286.1688088,619.8147123756866,13.947896499999999,225.8979325,1241.56851875
+0,326.978204225,708.2268137872856,13.9963715,227.913498,1238.5448918000002
+0,391.84397713,1258.0459333174051,13.97708,223.9180705,1052.6617701999967
+0,282.089933305,494.14902195573023,14.0310225,228.76641775000002,1042.7131751
+0,380.03002254000006,785.8124276577649,14.047008,228.648295,2041.64502655
+0,485.03607563499986,990.2036104031886,14.0435135,593.7413700000001,2384.9886236999882
+0,398.320053165,752.6652000831643,14.104851499999999,436.27744075,1264.501551899997
+0,645.08417883,1151.686949510218,14.310018,1026.114134,3258.2500845999984
+0,736.4698664800001,1378.054313440558,222.6983255,1027.77351125,3438.301208099989
+0,684.5808265649998,1159.0994518070354,224.29017,1027.72236075,3057.0754534
+0,576.8281072249999,923.2233779167972,14.5057325,1024.82361425,3040.4049964
+0,695.510851655,1166.6035242756757,223.902276,1028.02971375,3070.741363599997

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1/ntru_hps2048509.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0,13.424927695000001,0.7296674906330841,13.486661,13.896462750000001,14.3643828
+0,18.56953374,72.48588458852161,13.456068,13.908383,14.3724294
+0,15.70157449,20.833181308286132,13.496030000000001,14.072112749999999,14.6724166
+0,31.047633929999996,107.65978114216134,13.487655,13.81344125,14.519909
+0,61.29621911500001,200.9687184116353,13.6814275,13.902391499999998,224.3268397
+0,39.167061235,145.15699020078554,13.611416,13.94859675,14.954746299999998
+0,112.26421021,291.23019726645595,13.8285425,13.988345,1024.3676401
+0,126.37727095499999,302.48289016640024,13.875561999999999,14.07521275,1023.5500443999998
+0,117.78357489,311.22366295143576,13.8440645,14.025330749999998,1026.99059245
+0,187.69324614500002,493.6651446317939,13.895493,14.159061000000001,1033.86122355
+0,207.007606155,474.24261240960743,13.919796000000002,66.51640025,1037.3189292499999
+0,300.41075128000006,667.1326129047178,13.9309935,226.6094405,1044.4952610999999
+0,275.788902635,607.0404935846747,13.9232945,225.20102175,1055.407847399997
+0,351.73632254999995,754.1155850135956,13.948535,227.74217575,1238.9020676999996
+0,270.976762685,523.1593229512707,13.9940365,228.244619,1047.1322068999998
+0,406.261056295,875.2402266078209,14.0107045,229.172134,2029.8937693999999
+0,452.77451473499997,1252.6998692773757,14.0547375,438.67407825,1258.8181444499967
+0,446.764644,789.0679394094905,14.0586,584.6200690000001,2257.6451575
+0,430.9512827100001,767.3954954317977,14.0790155,491.69995275,2051.345687549999
+0,646.976798755,1437.591627915036,14.1530275,1020.621993,3060.5345450999994
+0,545.32308127,884.6661927237274,223.533679,1023.8312677499999,3044.165846
+0,841.08130498,1823.8519530011856,223.6913735,1027.3938745,3352.5043118999984
+0,682.2609063799999,1287.6764033212316,224.2910645,1027.34909075,3269.8221422000006
+0,734.00706806,1754.561749896278,224.463367,1024.57002225,3034.5896330499995

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1/prime256v1.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0,13.827628585000003,0.5995205511429136,13.788788499999999,14.06114,14.8050208
+0,23.84234618,100.97149768713983,13.931986,14.253470499999999,14.6383725
+0,36.24292924,143.05676717773272,14.081497500000001,14.53275525,15.127143299999995
+0,46.214347939999996,174.7451084814898,13.855561,14.1721245,15.229539099999997
+0,53.57633809,178.26422721191028,14.0296615,14.30623925,220.02200024999993
+0,62.728762085,201.04498461484482,13.999444,14.370615500000001,223.29340389999993
+0,102.57373194,267.2614391224484,14.040238500000001,14.518296,1027.852182
+0,146.09367861500002,378.5694925512625,14.374347499999999,14.66042525,1027.5409165
+0,180.37938259499998,416.0301646564367,14.550936,15.18071975,1034.30443395
+0,166.36319308500003,406.6517513086867,14.454373499999999,14.697659,1026.8572745
+0,231.98814242499998,569.5500145931867,14.515345,14.938358749999999,1055.4584387499967
+0,226.29370035,450.7513253786641,14.630727499999999,225.209883,1038.7620167999999
+0,282.430983065,709.3719714873032,14.5840615,226.08409125,1050.8256069499969
+0,304.73888442000003,549.9477766394089,14.637661999999999,230.25672874999998,1044.5582281999998
+0,313.860211585,716.5453659506071,14.62433,227.50925999999998,1044.10534495
+0,392.86383311500003,806.6631053504336,14.6578875,229.10678024999999,3045.49914695
+0,615.29455608,1400.6073551700342,14.6572975,1026.04511475,3180.100436549997
+0,420.324820865,782.5326593632882,14.6556,437.27182799999997,1677.3747040999933
+0,518.116970315,1047.927143548355,14.7243285,440.08217874999997,3050.8582883500003
+0,532.432604915,903.9010805145039,14.7542045,1015.4458425,3040.4536462
+0,683.167457635,1638.807186141608,223.843033,1023.8691672499999,3042.74382435
+0,618.04068259,1300.5348514097827,222.61648150000002,1024.69749425,3039.68156925
+0,660.0372597649999,1652.6995314986084,224.8889345,1023.886636,2109.763291249985
+0,723.6728406450001,2487.253191672756,226.0470585,1030.322423,2062.72082645

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/bikel1.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,14.93730817,1.6929052643339766,14.346944,17.144618,17.6230277
+3,20.673738795,71.88690856462078,15.40401,16.9052895,17.561482200000004
+3,26.195817355,102.0163042694707,17.027846500000003,17.46133,17.88591
+3,26.611730455,101.12326259751623,16.6101265,17.50107075,18.38357875
+3,62.415975575000004,210.43987311193834,17.339962,17.80913275,39.22593749999999
+3,64.93535402500001,216.1622055169738,17.415748999999998,18.032105499999997,49.2682264999964
+3,67.55094997,221.4645957338388,16.7801135,18.203281,87.93989264998339
+3,86.80872712,255.42090088579934,16.9651715,18.4735815,1029.67145245
+3,114.61233363,333.81033533574106,16.3588165,18.37824525,1030.06110605
+3,182.72633737,599.5056208498919,17.5846445,18.446274250000002,1041.8323758
+3,169.13736673,469.38639477714577,17.570562,18.4996595,1038.1858257499998
+3,172.94538186999998,396.8445488651344,17.605687,32.4348055,1044.6061201
+3,223.41953893000002,488.50117403221566,18.2223675,37.3184785,1053.6359696499997
+3,201.34628251499998,482.67500046791133,15.5042485,33.337431499999994,1049.1848319
+3,367.42173467,757.4125935599542,17.762928000000002,60.819696,2058.2431907
+3,393.53690415499995,944.6050941740635,18.441181,234.974779,1285.4380790999862
+3,433.20243917500005,1031.5086889514319,18.260409,358.76548775000003,1487.0580198999899
+3,372.66540047999996,782.9174170504751,18.225293,247.2631875,2047.0459642499998
+3,429.00885768499995,967.3119466626905,18.340927999999998,1014.7952105,1564.0373968999993
+3,444.67536000499996,1259.2042949846539,18.2562735,273.019518,1286.6721743499982
+3,708.67099868,1541.8926128159317,18.4416775,1033.673349,3516.95297069999
+3,782.2374863800001,2129.7100296105436,31.931765,1028.2164815,3079.573109199998
+3,862.615233345,1784.4625993979644,33.218842,1044.708501,3292.2360282999966
+3,828.8947287699999,2017.5604478545972,36.989406,1035.3647685,3277.3866692

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/frodo640aes.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,15.025669290000002,2.0863711359268766,14.0313595,18.07043325,18.50506005
+3,15.933120275,1.7117409476536773,15.1798185,15.96390525,18.4187409
+3,28.911661249999998,104.9777103048951,16.092502500000002,18.26685325,20.077089049999998
+3,35.09044696,107.78550544469849,18.198923999999998,19.35305725,37.738357399999956
+3,60.834644090000005,196.87949512290825,18.322487000000002,19.48060175,233.4353783
+3,65.92601925500001,305.52247194015615,18.422817000000002,19.33124625,233.52548850000002
+3,105.96326187999999,291.6018103729577,18.842927000000003,19.918564000000003,1032.5600151499998
+3,132.869134535,327.4387676361156,18.814010500000002,20.27965425,1038.16566645
+3,157.325125585,396.8369858497081,19.1831835,25.59904475,1044.5582107999999
+3,204.08813047499999,426.8504913512268,19.6552615,46.322433000000004,1071.6657604499967
+3,248.27556211000004,759.8591771058483,19.9970415,44.100353,1049.5708455000001
+3,254.246633355,494.7957398163003,19.921739,233.434443,1248.1414926
+3,295.89604396,1035.9282076606285,25.517861,232.7598405,1057.56908595
+3,379.91085365,666.8115759935223,33.2012925,315.69611999999995,1254.9863111
+3,313.744076425,610.1772757477906,31.1226265,240.4931365,1071.7006215499998
+3,284.9763962,560.6614438693956,32.551836,240.2988875,1256.9391123499997
+3,533.541111875,1448.5365805660401,42.098552999999995,507.2514175,1550.0453563999909
+3,535.27135362,966.9279566293452,51.8874485,1032.36614525,1644.5845973999749
+3,705.7503057450001,1043.9328167492743,240.260366,1049.67077925,2353.747718999999
+3,644.579947235,1136.7224577821296,233.16177249999998,1038.73510925,3116.888137699997
+3,1061.8786001099998,4597.003638725151,229.7098605,1056.4368319999999,3727.117108149996
+3,806.8798456500001,1643.4784960033205,239.42721699999998,1044.22450475,3074.66344485
+3,911.7264822399999,2523.9614021792936,245.472017,1059.80746225,3117.073042249998
+3,1225.5477734,2605.764475552003,266.16713200000004,1249.30320025,3998.915337749997

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/frodo640shake.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,15.502482904999997,2.455077405292077,14.845589,15.466228000000001,20.61938
+3,27.921698135,101.96969857909056,15.329572500000001,15.969016750000002,25.412767849999998
+3,26.943344065,78.67622316475843,16.170501,17.45019075,25.998782249999998
+3,47.973410709999996,162.51848500523712,15.1182305,20.95284625,39.22827569999983
+3,82.18021925500001,293.3440072229049,19.4230605,25.280894,241.49845999999957
+3,40.56853848,129.32640003786477,17.6767665,24.01995675,35.37853594999999
+3,100.56619239,263.51754243701436,22.004476500000003,25.538142750000002,1046.5459606
+3,112.442281425,301.125601886878,18.3367985,27.36058,1046.6328884
+3,159.079614535,384.75356651238013,26.348315999999997,33.289466749999995,1046.8059311499999
+3,211.411849985,1112.495360929331,22.782069,39.530874000000004,1042.42924755
+3,241.99965808000002,989.518772920973,24.880834,48.783393,1043.0956532999999
+3,238.21021825999998,482.4574258859285,27.1656765,234.20899275,1069.457507
+3,214.91850756,376.91415715760877,27.097977,236.728713,1061.0901686999998
+3,314.073736455,598.362840040462,30.866034499999998,251.03818325,1252.9986030999999
+3,401.97912350999997,720.0146285722554,33.076253,475.49788850000004,1307.4262238499969
+3,403.487294465,873.3679784275243,33.3136715,328.892157,1308.650600899997
+3,367.62347454,863.800180486718,36.911238,261.99032325,1459.5518859499998
+3,649.160155815,2062.993043280913,54.90158,1036.177756,2591.4616542999916
+3,533.551300995,940.3185890043378,55.435711999999995,1029.9474627499999,2209.8603523999977
+3,553.80076755,1284.9654641629913,82.80128149999999,1027.6311047499998,1351.5641217499942
+3,824.509123135,1956.987894617142,242.2091915,1055.3250074999999,2495.0680705999894
+3,1004.6649459050001,1975.2693980079637,259.0909565,1087.4257625,3550.3471546999904
+3,1345.68417342,4281.300191923928,256.8583855,1067.9180820000001,4481.157246449995
+3,1434.91614,4214.424886305993,255.7146485,1077.1944772499999,4608.456201749996

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/hqc128.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,13.861463925,0.8611711617018822,14.1042215,14.310569500000001,14.795688849999998
+3,19.388144245,72.0305831016561,14.1663695,14.54044425,15.108215449999998
+3,36.61131298,143.2988939616977,14.1669245,14.64397875,19.5424758
+3,30.677519609999994,125.12382945700475,14.3489305,14.95840725,20.605291749999793
+3,49.5164854,176.35616434957228,14.432457,14.75582275,38.60613179999999
+3,113.91059477,349.4608457627714,14.731924,15.1101755,1028.835887
+3,87.80559178499999,257.422231005383,14.46629,14.779791750000001,1029.237361
+3,65.570779985,202.73057591104865,14.641088,14.8222865,230.61380814999998
+3,137.80195329,316.678921577073,14.762960499999998,20.55796575,1039.6569261
+3,140.935424065,374.92331500046754,14.727989,23.84567775,1037.42149965
+3,149.28417448,320.56381625639915,14.749289000000001,34.5953685,1037.11217245
+3,256.6547067,1144.473076822286,14.8297655,40.22601525,1043.05565655
+3,255.23296279,512.5685781428269,14.881636,228.6222765,1046.8448263
+3,288.01322235,713.6856228072294,15.278138,229.97432700000002,1060.29590235
+3,464.83869663499996,2284.8289625591515,15.6876745,234.10961575,1244.1296440499996
+3,336.75171061000003,765.9908946251799,20.47269,251.5644005,1068.5009891499967
+3,444.3197446,820.2503009901246,29.615782499999998,774.7244792500001,2066.7744487
+3,637.6166359250001,3670.8776749909675,20.4999545,255.88245025,1310.6785286999925
+3,556.724191095,1124.6697624501583,35.960567,1024.53092825,3042.4363115499996
+3,477.072682615,1007.4997855469993,35.5280035,1015.5958055,2046.5765801999999
+3,637.10203615,1497.554588146348,37.138668499999994,1030.15072475,1898.1492236499998
+3,769.3359380099998,2717.43009883036,41.009039,742.70190825,3071.1776179499975
+3,682.1646049950001,1068.9683569352762,148.994857,1043.450945,3064.8533076500003
+3,792.1623067799999,1810.2579770921388,52.6150185,1045.50539025,2303.0858609999964

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/ntrulpr653.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,13.38649373,0.6867765859050723,13.532010499999998,13.7681495,14.19178445
+3,18.983095380000005,71.31100993825633,13.851200500000001,14.110916999999999,14.61072725
+3,14.54180462,14.551201589671399,13.4952145,13.9452825,14.8288153
+3,27.6192306,105.13953574359057,13.422933,13.782251,14.383715799999994
+3,44.751457665,163.36264191323775,13.7853645,14.0189765,37.41959339999998
+3,63.804127645,221.084699835606,13.923229,14.094825749999998,221.35011329999998
+3,48.959191835,182.6044890237729,13.702588500000001,14.000793749999998,34.38028084999999
+3,101.993832575,299.5516781939428,13.91921,14.1416575,1030.0542640499998
+3,114.75136871999999,354.3182699001297,14.014614,14.2116545,1027.3904065499999
+3,176.83461145500002,412.4077471298589,14.018051,14.231133750000001,1039.49544375
+3,190.75123676000004,528.6198721857537,14.0386505,14.3018915,1041.4968878
+3,266.79070630499996,652.1317918370754,14.1079595,35.64683975,1057.934567599997
+3,265.30670595,642.9599359874184,14.0892995,34.868737249999995,1044.3938070000002
+3,315.87159838,668.8550868542484,14.158837,227.92928874999998,1072.637887049997
+3,333.81984109000007,690.6026604778796,14.151676,229.3054325,1457.765065849996
+3,298.83654737500007,747.1433036561718,14.116550499999999,227.53899925000002,1257.7917663
+3,447.36530578,832.8262545359748,14.1828955,1016.1686567500001,2093.144038449983
+3,519.7681067350001,1434.6816353260394,14.1456105,343.62306325,2073.6265496499927
+3,611.45677054,1361.4640903920526,14.179024,1024.2439605,3041.227103
+3,734.818348235,1665.5019932495798,32.829399,1025.90238875,3268.531329299992
+3,564.381194715,1102.169184224065,23.274312000000002,1028.09070475,3039.6987725999998
+3,1053.10480853,5402.572765662321,41.236374999999995,1033.9355735000001,3080.771381099997
+3,720.29756973,1604.6540773059653,36.067706,1026.3414787499999,3266.15575455
+3,879.4182224749999,1649.2147223458535,36.4773005,1040.2995575,3365.3123996

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/ntrulpr761.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,13.52171641,0.7046480110215966,13.6089545,14.01627375,14.36979265
+3,18.794069555,72.21716327729213,13.7052265,14.00025775,14.383580949999997
+3,24.38403676,100.7380468031738,13.8985135,14.064855999999999,14.95920145
+3,20.656389869999998,73.56037465634365,13.5251155,13.931539,14.556954000000001
+3,33.02293110000001,126.75594259810792,13.5800545,13.9500335,14.453068049999999
+3,44.45957627000001,161.31027022132866,13.611864,13.89846125,45.9804750499969
+3,52.240100045,176.2010914113727,13.7132905,14.04282175,227.60965005
+3,109.49802885999998,288.4729280735098,14.006410500000001,14.18859775,1025.80188125
+3,139.674972515,372.06749089383226,14.006075,14.22702125,1030.31308725
+3,189.78609861500001,475.602677362095,14.018858999999999,14.2277635,1042.1043743
+3,227.72154245,689.8543777478986,14.038471999999999,14.24522625,1044.0921919999998
+3,240.90271079500002,585.2783142199919,14.142973999999999,34.539711000000004,1057.709388399997
+3,242.53317828,509.0268379998909,14.1328555,227.51180175,1041.5691114
+3,307.48598101,779.2742657986486,14.114151,226.91481124999999,1246.1249272
+3,369.14512271,929.7260095151642,14.147822,227.786457,1266.0346568999967
+3,530.5072349300001,1474.1704435296472,14.263183999999999,1021.83781,2065.863304299993
+3,390.03257433,764.622277414857,14.188589499999999,245.13566775,1260.2433772499978
+3,449.93083741500004,796.7699851385611,14.209236,1023.9162957499999,2042.47606025
+3,509.01262963,895.639058128479,23.4184195,1021.496195,2109.326237799983
+3,444.39291566500003,904.1116154624458,14.258011,905.67063625,1766.395038549995
+3,626.5718609099999,1880.444964040479,14.253765000000001,1023.77390725,2369.4589745999874
+3,690.3657414149999,1479.6722460566987,35.7394625,1026.5476595,3269.05958235
+3,1129.30911649,2709.6650402423484,34.569604999999996,1041.6531962499998,6533.49158449999
+3,861.5912550699999,1568.9717393312887,227.665924,1041.8779182499998,3362.145217199998

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/prime256v1.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,13.866693045,0.9360799401022344,13.932795500000001,14.49967475,15.07873015
+3,20.04188452,72.38098625471108,14.120696500000001,14.56580325,14.785346449999999
+3,27.248391655000006,104.51854987156024,13.882854,14.354389750000001,14.957957050000001
+3,31.121707594999997,124.47691939169795,14.073677499999999,14.36527375,14.951363099999996
+3,62.516780745,211.1090640806784,14.1970215,14.413138500000002,226.59692174999998
+3,50.03020229,163.92198729728756,14.178462,14.422346249999999,223.52230255
+3,82.15339320499999,230.20048823961986,14.438172999999999,14.65138825,269.06465284998654
+3,95.78786839,313.6813799241725,14.437822,14.6209535,1021.5200358
+3,113.81232172,327.40953448958186,14.382824,14.68130175,1023.59865135
+3,160.03567285,391.71189660453194,14.457317,14.660839249999999,1028.54877595
+3,205.15551215,448.99881769480794,14.536932,65.733083,1034.4273369999999
+3,195.56866463500003,456.62741893523133,14.430561999999998,14.789207750000001,1041.2131195
+3,283.849487545,693.7040050118035,14.569422,226.20323025,1041.3986227
+3,282.77672191,548.3039101768921,14.650203000000001,228.41568274999997,1041.5870777
+3,380.93188876,919.4665257381939,14.629261,228.948099,1257.31126855
+3,338.021323745,665.3194500810955,14.6480255,228.25540725,1501.1562892499903
+3,430.10945226999996,844.0807325331964,14.6760175,716.1046809999999,2038.176646
+3,426.88034837,734.9819761761457,14.6613345,1017.73554625,1448.2222080499998
+3,781.356412095,1916.2031876029082,14.811418,1024.3008682500001,3056.8257137999954
+3,573.65707443,1593.4791344485861,14.723032,1022.959422,3053.4902000499997
+3,561.174283855,1002.4299423799824,14.695949500000001,1023.9488584999999,3052.2498379000003
+3,762.5772556300002,1397.9359739069853,223.912612,1033.356664,3059.1649177499958
+3,681.5522725699999,1222.4860034091205,225.698109,1030.897445,3038.64092035
+3,648.3735753599999,1187.4759931715162,224.990813,1029.433412,2298.574994549987

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/sikep434.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,19.02196678,1.9577722054059816,17.946863999999998,20.18216,22.487405349999996
+3,18.179872225,1.0625729599138003,17.736993,18.361665499999997,20.201548250000002
+3,29.483537130000006,77.78154649563018,18.157715,20.1559065,40.31211795
+3,40.934565965000004,128.13607391415243,18.2242795,20.362495250000002,43.670021899999995
+3,74.52432559500001,205.2169207516867,20.472624500000002,22.671449000000003,244.43225839999997
+3,65.64926222499999,190.46694841881458,19.389809,25.240438249999997,239.93666584999974
+3,67.801824045,197.50208817254696,20.298885,22.81025125,237.3590233
+3,131.24242327,292.5997346217519,22.684247,34.988982,1041.3889497
+3,158.10896867000002,413.1800897514331,22.411772,37.67772600000001,1044.75750165
+3,217.994562745,611.5289675814223,22.156725,36.68643175,1048.9649871499998
+3,204.67144617999998,380.1105244044108,25.345797500000003,53.476881000000006,1051.21820525
+3,166.44952944999997,323.0910103100826,22.190192500000002,44.08720649999999,1046.22295715
+3,260.01004025500004,553.1973325487938,29.021586499999998,238.77094225,1074.566210349997
+3,281.00092463,519.5108575267678,28.36668,239.19418599999997,1062.1747627999998
+3,315.65524875000006,624.607240792836,25.159308,242.01095300000003,1259.3488082499998
+3,377.72254623000003,690.9243881545232,36.568573,257.65708425,1281.1564647999965
+3,401.66653689500004,1119.0803963018507,29.1673715,253.72939649999998,1261.4705414999999
+3,564.000605795,905.9100286726992,234.5085765,1035.2298795,2103.0135930499905
+3,427.464182625,692.9339526145214,44.5609965,890.3918474999999,1293.3703008999971
+3,580.588051195,1070.7844997339298,231.9725315,884.7518877499999,3069.2726209499997
+3,774.24272346,1952.8299927243925,46.338141,1035.30146125,3051.4385192500004
+3,711.5513497600001,2394.355272868512,43.6154145,1035.11587825,2086.1957952999965
+3,756.6151290900001,1401.6934174911503,240.217578,1037.04860275,3058.7840019499995
+3,832.191202815,1294.77177667657,245.200899,1050.8562192499999,3271.6477614999994

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/sikep503.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,22.395990395000002,3.852240287903906,21.672581,22.94502825,27.53035425
+3,39.75478361,124.95476986484101,21.383528,25.35026325,32.1898405
+3,33.64246195,102.17224642754041,20.320670999999997,22.778676,35.9233046
+3,49.25517507999999,159.53488450707871,20.586698499999997,23.24061325,30.897187549999995
+3,57.210169845,176.40724290716253,20.277041,26.743359,51.86286269999995
+3,86.426601625,232.9330472759112,21.7667585,32.16466375,1023.2279972499999
+3,108.958866495,337.7655168288309,22.4719325,29.469862250000002,1030.1087893000001
+3,131.48763292499999,298.40549309519344,23.7075545,34.099946,1033.0406497999998
+3,140.61900508,386.0719023856785,22.0375995,29.45942375,1040.19085675
+3,138.283237535,351.8645174987409,23.984204499999997,41.4731215,1046.4333665499998
+3,171.006032185,380.60505283786273,27.5042665,45.837466500000005,1044.5355671
+3,165.40961734499996,336.5695789302485,26.001132,48.006966,1048.3061843
+3,272.08122224,493.7032160417727,27.774929999999998,244.25909800000002,1063.7518406
+3,297.27448090999997,696.3904169025578,27.0596975,244.97147125,1067.7819375
+3,432.7463674250001,1135.8368531481,29.850908,446.0789815,1268.2626911000002
+3,467.09033371,921.7519752923996,34.5531105,944.6333864999999,1287.522688949994
+3,415.02671901499997,736.6888432521328,43.8051325,266.89618925,1836.760187949996
+3,448.25427761,829.7842742171282,34.9152995,1024.5252727499999,1464.9804287499996
+3,525.394026035,778.8305321037687,232.17057599999998,1039.7412055,2066.0038434
+3,471.296268655,904.2842965817389,41.643125999999995,722.763152,1487.2621312999945
+3,693.640727135,1175.5368381315038,239.45745699999998,1043.9198345,3067.9065717999993
+3,600.656888435,1302.7570908651505,50.9560055,1041.4650824999999,2261.3006273
+3,714.89490851,1383.1214771627303,41.741817499999996,1043.6454265,3364.730840899999
+3,806.333028135,1213.2472031495345,257.289389,1054.669124,3280.0914425999995

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/sntrup653.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,14.251125325,0.3870288479717622,14.162251999999999,14.2906155,14.89132395
+3,20.309472285000002,72.50491349065071,14.135881,14.777469,15.88925725
+3,19.57365835,70.85880505787256,14.2375095,14.838536750000001,15.938213799999998
+3,27.54039172,103.7110833593437,14.204722,14.708876,14.947211249999999
+3,76.90736697500002,316.95413525422583,14.598961,14.84115975,220.37420529999997
+3,59.85418766000001,201.27774090304,14.497292999999999,14.76144025,228.07334464999997
+3,82.147300575,247.29673543181283,14.604185000000001,14.881908000000001,1016.58170165
+3,121.30012147999997,370.50505199080095,14.710208000000002,14.90518625,1034.91314575
+3,121.84651050000001,312.6002948467215,14.7022955,14.91701875,1026.7425742
+3,167.326256705,463.2609156193847,14.73144,14.92781925,1036.58623335
+3,183.48670207499998,437.1707375056455,14.788424,15.086503250000002,1033.0218463499998
+3,217.467656365,646.5113902105048,14.7824675,33.833113,1040.88078135
+3,163.353979265,384.1461850727093,14.8192785,33.563019749999995,1035.3441876
+3,334.0629306149999,779.0449170329268,14.8999685,228.89998725,1045.6293362499998
+3,369.38607544,715.8399099058657,14.946257,230.75438724999998,1467.5264193999963
+3,424.80364983,821.1222289521919,14.921822,776.0424835,2048.44462025
+3,294.00474439,542.5623136821685,14.882569,230.56554624999998,1245.5236428499998
+3,577.719166885,1099.0441176966958,15.1942305,1024.62055775,3273.5871781499995
+3,411.16539805,830.3159816641847,14.918194,253.63518599999998,2050.9177472499996
+3,614.5934369299999,1512.1611636976465,15.0818075,1023.9183515,3058.1533042499996
+3,799.0028943000001,1462.4542582901636,35.637969999999996,1040.88441325,3753.837470699994
+3,786.51344262,1452.857846941392,40.8999865,1038.0595802500002,3454.7298254999987
+3,990.238002215,2627.587195101265,35.2471435,1037.71577125,3500.1600514499996
+3,744.20029858,1270.2579809622569,134.912979,1041.17365475,3360.8329463999985

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_alternative/sntrup761.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+3,14.341271965,0.5591694189353918,14.272649000000001,14.4194735,15.4620503
+3,22.187164974999995,74.87546136202563,14.3010925,14.42236525,15.267768799999999
+3,29.303735040000003,105.25595775332908,14.725737,15.31061225,16.497800299999998
+3,31.061486334999998,124.84454448682276,14.5802065,15.03110025,16.1518841
+3,58.68149643999999,200.41858222809515,14.773759,15.14124975,220.09285319999998
+3,37.940165109999995,129.67616841356264,14.684091500000001,15.1042565,43.8249006499967
+3,90.40875545,257.6105106773223,15.021571999999999,15.278591749999999,1015.882514
+3,78.178013625,237.59034489779725,14.904995,15.319283249999998,1015.0240888999999
+3,120.15605592,297.57474677664754,15.127818,15.3686315,1029.804983
+3,152.99039448,408.92787049907685,15.057162,15.453228750000001,1040.8590898500001
+3,121.04720148999999,323.1852638312938,15.038123500000001,15.37306225,1028.85636005
+3,215.41325015,630.9360862005122,15.246229,36.84560125,1042.5094501499998
+3,298.63748188000005,603.5276575119035,15.322765,228.1128445,1239.66994265
+3,335.7974282099999,719.3687526964517,15.2922625,230.65807174999998,1267.1095596999965
+3,337.78238108,716.8742989843524,15.318809,230.55259575,1236.1099431999999
+3,400.005300975,795.1315875393274,15.374072,246.21945975,2265.0491717999917
+3,401.74128741,802.3859496601144,15.339372000000001,247.285854,2051.65973105
+3,467.893880775,750.5418969120445,15.7817405,1022.43518975,1826.0905536999965
+3,683.9860312149999,1701.7009980523874,15.411416,1021.31032375,3278.666708349999
+3,550.68803364,1033.6490750222258,15.4649275,1026.6693135,3034.7642389499997
+3,458.9247007700001,859.1324389015266,15.424682,1019.1345382500001,1890.6712467499974
+3,725.423835225,2011.7735568827623,34.375864,1030.70691325,3057.9762895999997
+3,793.21518854,1708.0263081574724,227.6644705,1034.77124925,3075.3676443499967
+3,919.472868235,1602.329238457814,227.94065,1186.36724975,3465.9672733999996

pq-tls-benchmark-framework/emulation-exp/analyzed_old/results/corrupt/secLevel1_hybrid/p256_kyber512.csv

+id,avg,std,median,0.75-qtl,0.95-qtl
+0.25,14.181414035,0.8169085790046178,14.068562,14.696372,15.636068100000001
+0.25,19.281169470000002,71.17509075369708,14.288456,14.69600725,15.49239815
+0.25,20.267637249999996,73.682444725679,14.129198,14.5500195,15.354268350000002
+0.25,22.58375344,75.2227464600615,14.24784,14.66007225,15.277260899999998
+0.25,52.062334685,186.66092887903753,14.554969499999999,14.834573,17.370932849999683
+0.25,33.209548135,126.63966671429138,14.548769499999999,14.8679895,15.831138499999993
+0.25,79.41141325999999,310.25460755918266,14.480118000000001,14.85793475,222.50315235
+0.25,154.331921265,392.7025283153518,14.701278,14.966982999999999,1040.85932805
+0.25,135.40174212500003,315.517651061448,14.7340225,14.970572749999999,1026.8587226999998
+0.25,203.25599586,547.1918460493032,14.8231235,15.106874,1045.8441693
+0.25,139.85596612,391.9657405309961,14.7685105,14.972620749999999,1027.4893683500002
+0.25,223.367701955,537.7565672104856,14.892955,35.88213275,1044.0264678499998
+0.25,237.467475505,632.8364367168346,14.9189525,88.53611624999999,1044.1185721999998
+0.25,336.15555025000003,658.0301717672306,14.924792499999999,229.35542675,1290.5194327499903
+0.25,429.86967505499996,1424.4423066119198,14.967718999999999,234.49241175000003,1078.1978760499983
+0.25,403.25441314000005,916.7048729049645,14.941579,247.243474,1261.7178494499965
+0.25,356.454665325,715.0377428645684,14.904038499999999,229.44186675,1489.13463839999
+0.25,310.450885045,548.3510467129144,14.947491,248.06823075,1233.88619905
+0.25,593.30898256,1090.8181272889424,15.199343500000001,1027.6886265,3077.106286799997
+0.25,544.8831144200001,1025.6349973659421,15.041602999999999,1026.09693925,3037.75415385
+0.25,427.81145527500007,993.536301731688,15.021476,246.8660425,1832.3915999999963
+0.25,568.344984975,1049.7895488491276,35.2906905,1023.5161075,3041.3199514000003
+0.25,817.8130137649999,1551.4601818126564,228.735745,1038.855456,3739.59614375
+0.25,798.662467075,1748.6432324825819,36.118342999999996,1030.23756275,3332.0801889