5 replies to this topic

[so-appy]

Thanks in advance to any and all useful advice relating to this query...

To streamline and increase efficiency in the development and production
of a unique software item, each of which is designed to have a
similar code framework with a different and unique data package inside it...

Is it possible or how is it possible or in which programming language
is it easiest to be able to identify, in the executable of the compiled program,
the data space in which a data package exists, and then be able to replace it
with another piece of software...

ie. I have designed a program which has a small functional component and a
large, unique for each implementation, data component.
Rather than have to specify in the compile phase of each program or
rewrite the code and recompile or script a compile script for each instance of the program I wish to be able to compile the program once with a generic data package
and write another small program to essentially do a search and replace for the
data package.

This would increase the utility of the software considerably and
increase the efficiency of the construction and production.

Initial experiments with VB seem to indicate that data strings are encoded
before being stored, so a simple look up of a character string was possibly not
likely to work (however that was done with a rather antiquated version (V6) and things may have changed since then).

The data I want to encode is 8 bit data, all 256 characters possible, likely non compressible, and not stored in a compressed or coded format format
(ie. raw 8 bit data)

Can anyone offer any suggestions as to how this might be possible or feasible
or the best programming language that it would be feasible in?

I want each instance of the software to be a stand-alone, complete piece of software in itself, without the need for an external data file.

It seems intuitively achievable.
The data package sizes I am looking at range from 1M to 10G
(initially 1M to 100M for the prototypes)
Please let me know if you think (or are aware of) any issues with
executable file sizes of this magnitude or can see any functionality issues
relating to file sizes of these proportions.

Thank you in advance for your expertise and any time you spend
in helping to answer this query

Best Regards
so-appy

[Revolt]

Why the single file requirement if I may ask? What you wanted could be easily accomplished with a simple file that could be selected at runtime.

In theory you should be able to decompile the program, replace the space allocation code and recompile it but that would probably take even more time than just writing a script to manage the original source code.

The system used by trainers for pc games could also be useful to you. I'm unsure of the language most are programmed in (though a simple search might get you some info on that regard) but they somewhat monitor accesses to memory and replace values at the appropriate locations to change the money you have, speed requirements, etc. Check this out: Dynamic Cheat Manipulator | Download Dynamic Cheat Manipulator software for free at SourceForge.net

[Carver413]

most compiled languages are capable of embeding data directly into the program if thats what your asking. this would require recompiling of course. but the process should be rather easy. create a place hold file. or template file. create a script or program to inject the data into the template file. save new template and compile. size may be an issue. even if you had 16 gigs of ram there is no guarantee that you could allocate a chunk of memory that large. threre may also be compiler restrictions that would have to be resolved such as chunk size.

[Carver413]

A simple example of embedded data using Lazarus/Freepascal

program Project1;


{$mode objfpc}{$H+}


uses

  {$IFDEF UNIX}{$IFDEF UseCThreads}

  cthreads,

  {$ENDIF}{$ENDIF}

  Classes

  { you can add units after this };


{$R *.res}

Type

  TChunk = Array[0..3] of byte; //Define chunk size here.

Const

  a:TChunk =

    ($1,$a,$e,$f); //declare a chunk and fill with Hex$ for smaller size

  b:TChunk =

    ($1,$0,$e,$a); //declare a chunk and fill with Hex$ for smaller size

begin

end.

[so-appy]

Dear Revolt

Thank you for taking the time to reply and consider my query.

There are a few reasons for it being a single file application requirement not the least novelty, innovation, ease of use, user interface
and when multiple versions of the software exist on the same system to avoid errors and problems and security to name a few.

Yes, it would be a trivial implementation if the data file were a separate file, but that is not the plan for this concept.

Decompiling would take more if not as much time as recompiling and is prone to errors and in the case of this particular software product is
not really an issue, but may be useful to consider for other projects so thank you for that advice and info...

Scripting a recompile with a search and replace for the data include is the standard programming method, but time is money
and for thousands of potential compiles this could represent considerable compilation commitments on resources I will want to be
using for other projects, and also, if a third program is able to manipulate or add the new data directly it expands options for web based distribution
and frees the distribution platform...

Thanks for your advice.

Best Regards
so-appy

[so-appy]

~Thanks for taking the time to respond carver413

I am aware of a variety of options in various programming languages for including data or blocks of data
and the standard programming solution to this issue would be to write a compile script which
either did a search and replace of an include file name or a search and replace of a dataset filename or a search and replace of
possibly the entire dataset itself in a text encoding in the source code code, and then a command line compile
of each version with a different output filename (which I think you were alluding to).

For potentially thousands of instances of the program however, with large datasets, this represents a significant time investment
and would possibly waste system resources that could otherwise be doing more productive things, and, as you mentioned, memory requirements
for large compiles may be an issue, but would probably not be an impediment or limitation for a file based search and replace of a dataset.

Further, many programming languages encode embedded data in various ways (sometimes compressing it sometime applying bit packing schemes etc...)
meaning that the raw code is not directly search and replaceable, so hex editor replacing (or standard text search) doesn't really work there.

For innovation and efficiency reasons, and for a number of useful spinoffs from the development of such a process
(and as mentioned elsewhere in the thread to liberate it somewhat from the platform dependancy and to be able to do JIT delivery
on a number of current eCommerce type platforms (and future platforms) I am wanting to have the initial code
so that there is an identifiable block of data in it either at a location whose co-ordinates are precisely known by me
or whose content can be identified by a third program, and then do a search and replace.

If one instance of the compiled program were able to fulfil those requirements I could write another program to do the code replacement
in any language (or via scripting such as pearl etc...)

It seems intuitively doable, my query is in terms of .NET, C#, VISUAL STUDIO etc... and any other modern or current Windows platform development
languages and systems, what is the easiest and most likely to be able to achieve or accomodate this.
I was hoping for advice on the VISUAL STUDIO suite variants and similar as the shared framework for each of the software components
will have graphic elements that I want to have the look and feel of the modern WinXP and Windows7 dialogues.
However I am open to other programming languages and development environments, game platform development etc...

~Thanks again for your response.

Best Regards
so-appy