java - Proprietary HTTP/1.0 TCP protocol trouble -

i'm making attempt write open source alternative solution proprietary piece of server software called enterprise service manager using java. on suite of proprietary software contains several client software programs communicate server using hacked http/1.0 protocol. end goal have java server emulates esm , existing client programs working it, i'm focusing on 1 particular client program instigates ssl/tlsv1 session later on in chain of connecting.

using tcp proxy able determine initial data sent client server. first bit of data "get /fwd/esm/xclientcontrol http/1.0" , nothing else. client waits server respond "http/1.0 200 ok" "content-type: application/octet-stream". client responses mumbled mess of binary:

0000000 0316 0001 0137 0000 0333 5301 7a48 6cf5 0000020 6c2c 077a 90c9 12c2 9c4b d182 b124 c3d0 0000040 adce 3eec ed7a e787 1d6d 0099 0600 0500 0000060 0400 0300 0001 0400 2300 0000 0316 0001 0000100 1086 0000 0082 5a80 84c2 f401 729b a29c 0000120 a872 8f1a 6622 70b9 bbcd bbf4 e0e6 18df 0000140 1592 c00d 37ab e555 36d3 8c5c 6e6f 5ae2 0000160 90b7 9bf6 b9b6 7c49 9f01 bd0f 6327 c682 0000200 0bc1 764c aff1 2de7 e4a3 35f8 c640 2757 0000220 cf8d bd69 3f51 5138 9b44 9f2b 5ea7 175d 0000240 17a8 de1b 88d9 7365 569c 09d5 2305 0fbe 0000260 1cc7 f68a 5465 7d1e 7b14 bb94 b0b3 9524 0000300 a873 8e32 5fc3 1452 0103 0100 1601 0103 0000320 2400 c269 15c2 ec42 d03d ced0 d932 3cf8 0000340 cddd 4c9b 0a51 ce15 b1f9 091d bf79 ad98 0000360 3b7a db75 09b9 0317 0001 9d57 94c5 c266 0000400 a8e7 9615 c332 b48b 102e ece7 72f7 bb27 0000420 9999 ec09 6a0e 3637 0ad9 49c0 1814 5658 0000440 eb1e 74c5 446f 4b63 9efa 8e1a a377 fcf1 0000460 45de afde 8bd0 a70c d8f6 55fa 77ca a113 0000500 455a f375 f6ed 0cc9 fc2d 3232 1984 dc7e 0000520 0c2f 0000522 

then server responses with:

0000000 0316 0001 0230 0000 032c 5301 7a48 85dc 0000020 f105 3587 b793 48f3 d20b e82f 5e5c 8010 0000040 49e0 b5af 12d1 80b0 be0f 00e9 0500 0000 0000060 0004 0023 1600 0103 d701 000b d301 0100 0000100 00d0 cd01 8230 c901 8230 3201 0102 3001 0000120 060d 2a09 4886 f786 010d 0501 0005 2e30 0000140 2c31 0830 0306 0455 130b 3101 0a30 0306 0000160 0455 130a 4503 4d53 1430 0306 0455 1303 0000200 450d 4d53 3331 3639 3336 3435 3030 1e30 0000220 0d17 3431 3430 3430 3831 3431 3034 175a 0000240 310d 3035 3034 3134 3138 3436 5a30 2e30 0000260 2c31 0830 0306 0455 130b 3101 0a30 0306 0000300 0455 130a 4503 4d53 1430 0306 0455 1303 0000320 450d 4d53 3331 3639 3336 3435 3030 8130 0000340 309d 060d 2a09 4886 f786 010d 0101 0005 0000360 8103 008b 8130 0287 8181 ea00 f6ce 9d86 0000400 4fbc f0a1 e7d4 2e16 2afb 1fce 5d02 f99a 0000420 934b 1a60 39d2 2060 8b77 7288 758b d3cf 0000440 2105 2c95 2bcc d455 4d2a aa81 1071 ef70 0000460 ebce 51a2 f38a 3b67 5014 5111 e42c 7ea7 0000500 e007 272b e9a2 eb42 5903 3366 49c7 d1d7 0000520 33d3 ea17 fbd4 51d4 1ff6 1a1a 0fea 86fb 0000540 9b00 32bf fa72 0530 05ba ae17 18c0 57cd 0000560 f769 4472 2a2e 5e04 6550 02bf 0301 0d30 0000600 0906 862a 8648 0df7 0101 0505 0300 8181 0000620 7500 a884 6b80 9031 34dd b6c1 0112 6c4b 0000640 7dd6 8c40 9e13 7113 f673 b832 c9e5 9bdb 0000660 8c00 faa3 6106 ea90 1239 c9c7 01bb 54e3 0000700 70a8 e50b 8a9c 8e3b a89b 848f b87c a579 0000720 2e9b 3c53 f4df 2fe3 065f 9c78 07f9 d4ae 0000740 e334 f56e bda3 3ce0 5e74 8af5 a46c fbc4 0000760 9056 6fab 9eda dcac 87bd a61d 0a32 deed 0001000 9f0b a8b5 7f7b cfca d50d 360d 463b 3ffa 0001020 1656 0103 0400 000e 0000 0316 0001 04aa 0001040 0000 00a6 0000 0000 b2a0 4b6e d643 2d5b 0001060 f957 150a cf69 cab7 f5ff d5d0 f7c1 4760 0001100 5980 c446 1cc5 36ed 6d3c 66e9 210c ecf2 0001120 c39e c4d7 ef1e 2afb e0fb 5a56 65ee 9701 0001140 85da d3fb 4f0e 5691 6828 7ed1 0f5d b6ff 0001160 1d09 db27 d599 f0de 2482 91e3 d957 51d2 0001200 aab7 8035 8c31 1ce6 4670 3196 9a4f 49df 0001220 a8cc cd63 a847 7ee1 64c5 13ea fcef 2601 0001240 5355 5574 65bb 41b6 6eff 34d5 fcf7 0987 0001260 2e74 1b31 2ae6 cffe 78ed f5b5 1b42 290a 0001300 4742 c61f c92d 2635 14ec 0103 0100 1601 0001320 0103 2400 0af9 71ac c895 1b03 c35d 74b5 0001340 6f19 7bf9 c919 fb91 0766 701e 3126 b652 0001360 78f9 ded6 abb2 1071 0317 0001 c4b8 af59 0001400 d41e 416a 3286 2f83 18f2 a0da c561 b81e 0001420 d173 ec57 e141 0833 9579 8275 f93e a3e8 0001440 4dff 1986 1ed9 6509 7129 5c66 1687 495e 0001460 6e55 b789 48bf 56e0 44c6 4807 8a81 659c 0001500 47b7 22c8 6ba7 846c a5a5 3f39 2d60 f711 0001520 3812 a9c6 e007 73b7 40ca afd5 8671 21c9 0001540 71dd 3d6c 5f45 3e81 e45a 7665 4326 d557 0001560 e327 c1b6 ffc7 22e7 5735 d5d8 4e13 0e02 0001600 f6da d03a db98 1b3b f3c3 2efd d966 d22d 0001620 6fc5 b359 89c7 628e 4d8b 09b3 fc1b af81 0001640 0981 53f0 0deb 5a82 316e 7634 c706 787d 0001660 3910 ed8c 0099 0001665 

then closes connection.

after client makes few more requests in 1 of them ssl/tlsv1 handshakes. i'm still unsure if can past first request more includes determining ssl procedure hacked well.

i able replicate initial connection esm using simple nio client in java sent original block of data.

putting aside long road ahead make work, few questions have be, see way can make data legible? see hint of encryption? binary code be? able find 2 different certs on esm server, 1 keystore titled "esm_ssl_certs" , other appears me x.509 (i might wrong) cert organization, organization_unit, certificate , key. there ssl encryption establish later self-signed cert.

i'm lost need research , i'm not sure need asking. appreciate more knowledgeable me on subject can give. if need be, can post more details upon request. thank in advanced.

here ideas:

• try looking @ data in hexadecimal.

• try saving bytes file , running linux file command on see if can identify file type.

• try decompression tools on it.

• try comparing bytes seeing ssl negotiation ... based on spec says.

if ssl encrypted data, in difficult, unless can find out initial keys are.