SourceForge.net: SF.net Project News: FileHelpers Library (including full news text)

FileHelpers 2.0.0 Released

marcosmeli@users.sourceforge.net (Marcos Meli) — Tue, 10 Apr 2007 04:45:45 GMT

Here do you have all the stuff that keep me busy for a while :P

Here is the download link:

http://sourceforge.net/project/showfiles.php?group_id=152382&package_id=169468&release_id=499968

And the History en HTML format with links:

http://www.filehelpers.com/history.html#v2_0_0

Breaking Changes
----------------

The constructor of the MultiRecordEngine was changed to allow params args, and to allow users to dont pass the RecordSelector (in write operations were dont nedded).
EndsRead and EndsWrite deleted, now you have a Close() operation to simplify the API and to make it similar to the System.Data namespace. (The async engines also implement IDisposable)
Rename FileHelperException --> FileHelpersException a better name =) and an easy to solve problem
By default the numeric fields in the FixedLengthRecords that dont have an [FieldAlign] will be aligned to the right and the rest the left by default, with this we avoid the problem of generate files with different meaning when read and write

Performance Related
-------------------

Internal use of Reflection.Emit, Dynamic Methods and char buffers to get more than 50% enhace in performance and for .NET 2.0 more than 65%
The Read..AsDT methods now create the DataTable record by record, not at the end, so you can handle large files without any memory overload
Not more reflection in the operations (only in the constructors).
FieldSorter faster too, Removed REFLECTION for EMIT

Shining new features
--------------------

The very cool DelimitedFileEngine and FixedFileEngine that allow to change options at RunTime (also with generic versions) Check an example here
ConditionalRecords you can easily include or exclude certain records based on a RecordCondition (like BeginsWith, EndsWith, Contains or RegEx) Check the example
A new GenericDatabaseStorage excellent to allow others Db with ASO.NET support to work with the FileHelpers (Thanks Rodolfo Finochietti)
Experimental Mono Project Support (some users are already using it, we hope that in the next release we have a completely working version for this framework)
New Version Checker in the demos and Wizard (both with renew look and feel) Check the movies
.NET 2.0 Nullable Types support. (Thanks Vijayan) Check the example
Notification Interfaces: you can now get notified of the events simply implement
Enhaced Debugging in .NET 2.0 with DebuggerDisplay and DebuggerBrowsable, DebugVisualizars to come.
The Wizard has a Record Class Test that allow you to intruduce sample data and your class and check for errors or results.
You can check some screencast about the demos, wizard and other features of the library.
Now the code for Vs2003 and Vs2005 are both on SVN, sharing the same files and the build script autodetect your Visual Studio installation. So no more excuses time to get involved =)
The engines with async ops are now IDisposable and IEnumerable so you can declare it with using(..) and use them in a foreach loop

API Changes and Extensions
--------------------------

The write methods are less strict you can pass now an IEnumerable instead of an array, so you can direct pass a List or ArrayLIst to the method without need of do a ToArray()
RunTime records can now get a DataTable in the constructor and so use DataColumns to create one field for each column.
Generic versions of near all the engines of the library (is really hard to mantain the two copies :P so every one is time to port !!)
Async Operations in others engines like the MultiRecordEngine
Event support to the MultiRecordEngine thanks to the contribution of Francis de Fouchier
The library handle now infinite levels of inheritance adding the high level fields first in the result record class.
Allow to ignore spaces and tabs in the IgnoreEmptyLinesAttribute
IgnoreCommentedLines thanks to MCampbell
The Read methods now have a maxRecords args to tell to the engines how much them must read.
A new constructor overload to the engines that allow to pass the Encoding useful to make the users aware of the Encoding feature.
FileTransformEngine with two new methods ReadAndTransformRecords and TransformRecords
Flush Method to AsyncEngines to allow users to ensure data writte for example if there are using the lib for logging.
The converters are now smarter they validate to wich types can be assigned and throw exceptions if is wrong (for example if you use ConverterKind.Decimal in a int field)
The ConvertException has a lot of context information: LineNumber, ColumnNumber, FieldName. Also better Exception Message.
The integer converters recieve also now a decimal separator to build a InvariantCulture to format and parse the values and strings

Small Changes
-------------

Fixed the problem with ASP.NET: "Line 0: Metadata file 'filehelpers.dll' could not be found"
Fixed some problems with the CsvClassBuilder (is a good practice to dont release the last minute added features)
The Assemblies are now Signed
The new ConverterBase.DefaultDateTimeFormat to avoid set the converter format field by field =)
Some error messages from the library were rewritten and now provide more context information (line and column numbers, field name, etc).
ExcelStorage support for more than 26 columns (thxs to Mark Izendooren)
Encoding Support to the CsvEngine
The wizard remember all the paths now
SkipThisRecord in the AfterReadRecordEventArgs, this allow tho skip records from the results (thanks Crestline)
More Methods for the CommonEngine
More than 420+ NUnit tests !!! (I cant belive it, I´m a lazy developer but I know how much worth to test the lib)
Now you have two versions for the demos, one for each framework version.
The FixedLengthClassBuilder has more constructor to set the lengths of the fields in you instruction
Improved docs. (Thanks Antoine and Matt)
A lot of new examples check it out !!

Happy Coding
Marcos (0 comments)

New FileHelpers Dev Blog

marcosmeli@users.sourceforge.net (Marcos Meli) — Mon, 11 Dec 2006 15:12:43 GMT

Hello Everybody

I´m alive and coding again =)

I´m trying to do the last testing and coding of the version 1.7.0 I hope to finish it before the end of year !!

In the meanwhile you can check out the new blog to share ideas about the development of the library.

http://filehelpers.blogspot.com/

Happy Codding

Marcos (0 comments)

Domain www.filehelpers.com up and running =)

marcosmeli@users.sourceforge.net (Marcos Meli) — Thu, 24 Aug 2006 23:29:23 GMT

Thanks to the donation of Antoine Marie (www.antoinemarie.com/) , just for my birthday, we can use now the new domain:

http://www.filehelpers.com

So the addresse of the forums is http://www.filehelpers.com/forums

I´m very happy with the donation, THANKS ANTOINE

Remember to send any feedback about the library.
Marcos (0 comments)

FileHelpers 1.6.0 Released

marcosmeli@users.sourceforge.net (Marcos Meli) — Wed, 16 Aug 2006 22:53:15 GMT

Hi All !!

The library has 10 months now and a lot of things were added. So I want to thanks you all the people that send your feedback, post motivating entries on the web and a lot of other ones that help me in the development. (partial list of contributors)

Now the important things !!... RunTime Records are here thanks to some help Gerhardt Scriven to use RunTime Compilation. Enjoy it !!

I think in this version like the 2.0 version because the RunTime Records were one of the most requested features and were one of the most hard to implement features, A lot of effort was put in the usability and in a clear API, if you have any suggestion please post it in the forums =)
Other important news are the CsvEngine (for generic Csv files), the FileDiffEngine, the MultiRecordEngine,..

Breaking changes:

- MOD: Now the FixedLengthRecords must contain the exact number of chars. You can pass an extra parameter in the [FixedLengthRecord] attribute to allow a different behavior.
- MOD: CommonActions renamed to CommonSelector in the MasterDetailEngine

Mayor changes:

- ADD: The awaited RunTime Records thanks to the suggestion of Gerhardt Scriven to use RunTime Compilation. Supports for classes written in C# and VB.NET
There are a lot of features inside this new feature, so take a look at the docs
- MOD: The Record Wizard was completely rewritten to use the run time records and now has a lot of options to check the classes the you are building, in the future the wizard can help you to discover the record class based on a sample file
- ADD: Generic Csv files are now supported (for example Excel generated ones)
Was a hard work but worth it because it can be used with any engine.
In the next version I´ll add the Excellent Fast CSV Reader from Sébastien Lorion for standalone CSV processing.
- ADD: Event support !! from this version the FileHelperEngine has 4 events 2 for read operations and 2 for write that are throw before and after process each record.
- ADD: FileDiffEngine to compare files with the same record layout.
- ADD: MultiRecordEngine to read files with different record layouts.
- ADD: DataTableToCsv, CsvToDataTable, RecordsToDataTable, RemoveDuplicateRecords and other methods to the CommonEngine
- ADD: FixedMode Enum for fixed length records, used to provide better validations in Fixed Length Files
- ADD: Template support in ExcelStorage
- ADD: A lot of new examples check it out !!
- MOD: The decimal, double, and single converters now recieve an string param to set the decimal delimiter (by default is ".")
- MOD: Improved docs in the SDK, the general help and examples. (Thanks Antoine)

Minor changes:

- FIX: FieldDelimiter don´t work very well after the refactor of the previous version.
- FIX: Bug when copy or saving to file in the Wizard
- DEL: more and more properties of the internal classes, this enhance both the performance and the dll size.
- MOD: A lot of internal refactor and enhaces
- MOD: ExcelStorage now return the cells values when using ErrorMode.SaveAndContinue
- ADD: These excellents VB.NET - C# comparison cheat sheets (to avoid translate the examples)
- ADD: RecordsToDataTable in the CommonEngine
- ADD: A lot of options to CommonSelector like: MasterIfBegins, MasterIfEnds, MasterIfEnclosed and the details counterparts
- ADD: BooleanConverter can recieve two arguments for the true or false values
- ADD: New constructor to the SqlServerStorage to direct pass the ConnectionString. Thanks to Anatoly Kleyman
- ADD: Again a lot of new NUnit tests near the 320 (wow I never think in write so much)
- ADD: Library Statistics page.
- ADD: How to help to the Library page.

Happy Coding
Marcos (0 comments)

FileHelpers 1.5.0 Released

marcosmeli@users.sourceforge.net (Marcos Meli) — Mon, 26 Jun 2006 15:18:04 GMT

With some delays but the library is always moving forward thanks to a hard work and the help of all you.

Breaking changes:

- Both DataStorages: SqlServerStorage and AccessStorage completely changed, from now you don´t need to override this classes (in fact you can´t because are sealed)
You can use these classes as client and provide the callbacks for the Insert and Fill record methods.(check the examples)
- The FieldQuoted attribute was completely rewritten to support optional quote for read, write or both.
So you need to use now the QuoteMode enum and not only a boolean. (For Excel generated CSV files you must use the QuoteMode.OptionalForBoth value) (check the examples)
- MOD: DataBaseStorage to DatabaseStorage (Case change)
- MOD: Extract and Insert Records now returns the inserted or extracted records

Mayor changes:

- ADD: New FielOptional that allows that the n last fields of a record be optional.
- ADD: Multiline Quoted Field support (with this we finish the support for Excel generated CSV files)
- ADD: Multiline Records supports via the FieldInNewLine attribute to allow records to span multiple lines
- ADD: OleDbStorage thanks to Permjeet Pandha to use with any Data Storage that supports OleDb access.
- ADD: The ReadMethods of the FileHelperEngine can return the data as a DataTable the sufix of this methods are AsDT. That is perfect to load sample data into your NUnit test (examples comming soon)
- ADD: New IgnoreEmptyLines that allows you to mark a record with it to ignore empty lines in the source data.
- ADD: CommonEngine.SortFile and SortRecords by field or using the IComparable interface if the records implement it.
- ADD: New demo code in a Console Application project to make easy to newbies play with the library. (download it from the examples)
- MOD: A lot of refactoring in the source code and changes in the the folder layout.
- MOD: Code in the Docs now highlighted with the excellent dp.SyntaxHighlighter from Alex Gorbatchev

Minor changes:

- FIX: The AsyncEngine could close the File in the BeginWriteFile under some conditions.
- FIX: IO Exception when appending to an empty file.
- FIX: Syntax in some samples and in the Wizard Templates.
- FIX: The notify progress in the DatabaseStorage working fine and tested.
- FIX: A little issue in the DatabaseStorage that can throws InvalidaCastException in the constructor.
- DEL: A lot of properties in the internal classes were deleted (aiming to reduce the dll size and library performance, much better for Compact Framework)
- DEL: The obsolete LastErrors of the ErrorManager. Now is mandatory to use Errors.
- ADD: More and enhaced NUnit tests (more than 260) based in the coverage of clover.net
- ADD: Some new and corrected Usage Examples.
- ADD: Now the Wizard Supports FieldOptional and IgnoreEmptyLines, also you can generate the Properties that maps to the fields (useful for Data Binding the RecordClass to any binding aware control in VS2005)
- ADD: Encoding support for source and destination in the transformation engine.
- ADD: A new feature for the custom converters to allow them to handle the null values
- ADD: More features in the CommonEngine: TransformFile and the fast TransformFileAsync
- ADD: two Easy methods to export/import data to/from a file.
- MOD: Wizard: now use the Beta2 of the excellent FireBall CodeEditor.
- MOD: A lot of doc corrections and adds (my english is now a bit better. I learn english with the write and fix schema =)
- MOD: Demo App rewritten to show some new features and show the syntax changes in the old ones.

Remember that the library is moving forward thanks to the help of all you. Please don't hesitate to contact me and send your ideas, question, problems or any other feedback (0 comments)

FileHelpers 1.5.0 Closed beta

marcosmeli@users.sourceforge.net (Marcos Meli) — Tue, 13 Jun 2006 04:07:15 GMT

I´m really close to the new 1.5.0 release, I´m in the task of rewrite some docs and test the new features.

I need a bit of help to Review the docs and get the last time feedback about the new features to correct or add all the small things that you found useful.

I think that you´ll be happy with all the new features and the improved quality =)

Any body interested in help in this task send my a mail to marcosdotnet [at] yahoo.com.ar or post a message in the support forums at http://filehelpers.sourceforge.net/forums/

Thanks a lot for your time and help
Marcos (0 comments)

FileHelpers in Level Extreme Magazine

marcosmeli@users.sourceforge.net (Marcos Meli) — Sat, 06 May 2006 22:02:26 GMT

You can check the last article about the FileHelpers in the May issue of the Level Extreme magazine at:

http://www.levelextreme.net/wconnect/wc.dll?10,7,10,52

Full issue here:

http://www.levelextreme.net/wconnect/wc.dll?10,7,11,May2006

Feedback is welcome :) (0 comments)

FileHelpers 1.4.0 Released

marcosmeli@users.sourceforge.net (Marcos Meli) — Sat, 18 Mar 2006 00:25:10 GMT

After a lot of work I release it !!!

Mayor changes:

DEL: ConverterKind.Custom was removed bacause was irrelevant to use custom converters now the lib use [FieldConverter(typeof(YouConverter))]
DEL: Some Exceptions trying to simplify the Exception handling and reducing the number of differnt exceptions. The deleted exceptions are NullValueException, QuotedStringException and InternalException all replaced with the BadUsageException.
FIX: The Async engines now uses the ErrorMode in all cases.
MOD: The MasterDetailEngine now takes the IgnoreFirst and IgnoreLast options from the master record an not from the detail.
MOD: All methods in the FileHelpers Engines that returns bool changed to void because they was using error handling with exceptions and a bool result is redundant and useless, this provide also a more clear OOD.
MOD: LoadErrors from the ErrorManager class now is static.
ADD: The CommonEngine: with four static methods for the common operations.
ADD: The Progress Notification feature, with it you can get notified of the progress of each opration.
ADD: Full ExcelCSV support, a new parameter to the FieldQuoted attribute to indicate if the Quote is optional (like excel does with CSV)
NEWS: Full Subversion support in Sourceforge: now you can help in the development using any subversion client and send me the patchs with your changes.
https://svn.sourceforge.net/svnroot/filehelpers/trunk

NEWS: FileHelpers Forums the main please for your feedback and discussions about the library.
http://filehelpers/sourceforge.net/forums/

Minor changes:

DEL: Description property to the FieldBase this is really old (from the beggining of the lib.)
DEL: Demo zip file now is merged into the binary zip.
MOD: EngineBase.LineNum and ErrorInfo.LineNum now are LineNumber.
MOD: Enhaced exception messages.
MOD: SaveErrors can recieve a header string to write into the file.
MOD: ErrorManager.LastErrors renamed to ErrorManager.Errors
ADD: The ExcelStorage checks if Excel is installed, if not throws a BadUsageException
ADD: More NUnit tests (more than 185)
ADD: A lot of new Usage Examples.
ADD: A new form in the demo app that shows how to use the ProgressNotification.
ADD: Wizard: now use the the FireBall CodeEditor for syntax highlight.
ADD: Wizard: a lot of new templates and fix in others.

To enjoy the library.
Regards
Marcos (0 comments)

FileHelpers Forums

marcosmeli@users.sourceforge.net (Marcos Meli) — Fri, 17 Mar 2006 14:53:24 GMT

I´m proud to announce the new FileHelpers Forums at

http://filehelpers.sourceforge.net/forums/

In this place you can post your support questions, new ideas, bug reports and everything else.

I think that the forums are a better place to discuss about the library then I keep sending mails to the list but the forums be the main place to share your ideas and problems.

Check it out
Marcos (0 comments)

What add next to the FileHelpers ?

marcosmeli@users.sourceforge.net (Marcos Meli) — Thu, 02 Mar 2006 13:42:57 GMT

Hi All !!

I´m comming back to the work and adding some new features to the library.

This are some of the next steps, I need your suggestion to know what is the most useful or what do you want that I do first:

- IgnoreIfContains: an attribute to ignore lines that contains some string.

- FileTransformation: to change from Delimited to Fixed or from CSV to Tab delimited

- Template Support for ExcelStorage

- Progress notification: An events in the engines that can be customized to fires each n% or n records. (adding my XpProgressBar =)

- Diff Support: an option to compare two datastorages and generate a file with the differences.

- Enhace the Wizard.

- Header Footer Engine: a new engine to support different records for the header and for the footer.

Left your comment in the CodeProject at

http://www.codeproject.com/useritems/FileHelpers.asp?forumid=231202&mpp=50&select=1391222&df=100#xx1391222xx

or in the mailing list.

To get in send a mail to: filehelpers-general-request@lists.sourceforge.net

with the subject: subscribe

Cheers
Marcos (0 comments)