1 reply to this topic

[Master674]

Hi,

I have a small problem with my RC4 Function. I'm using an __out parameter to get the output string.

My function looks like this:

void rc4( const char *str, const char *key, char **output )

{

	char *temp;

	int i,j=0,t,tmp,tmp2,s[256], k[256];

	for (tmp=0;tmp<256;tmp++)

	{

		s[tmp]=tmp;

		k[tmp]=key[(tmp % strlen(key))];

	}

	for (i=0;i<256;i++)

	{

		j = (j + s[i] + k[i]) % 256;

		tmp=s[i];

		s[i]=s[j];

		s[j]=tmp;

	}

	temp = new char[(int)strlen(str) + 1];

	i=j=0;

	for (tmp=0;tmp<(int)strlen(str);tmp++)

	{

		i = (i + 1) % 256;

		j = (j + s[i]) % 256;

		tmp2=s[i];

		s[i]=s[j];

		s[j]=tmp2;

		t = (s[i] + s[j]) % 256;

		if (s[t]==str[tmp])

			temp[tmp]=str[tmp];

		else

			temp[tmp]=s[t]^str[tmp];

	}

	temp[tmp] = '\0';

	*output = temp;

}

Call:

char *decrypted;

rc4( "data", "key", &decrypted );

So I'm just wondering if this is legit and can't cause any Buffer Overflows. Because some of my Webserver -> Client decryptions fail - Almost 50%!
The result RC4 is sometimes cut at some random location. Until this location it's decrypted fine. And sometimes it just works fine.

I thought of maybe some memory leak...

[fread]

I tried running this code on windows. For some reason the anti virus thinks its dangerous and kills the execution.