Commit graph

56 commits

Author SHA1 Message Date
snipe 3ffa3534a0 Use environmental variables in memory limits
Signed-off-by: snipe <snipe@snipe.net>
2021-04-05 21:47:57 -07:00
snipe 052f8e2c42 Addresses - not fixes - #8740
More investigation here is needed
2020-11-13 12:34:42 -08:00
Martin Meredith e3e0d57f56 Minor code cleanup bits and bobs (#6805)
* Add IDE Helper files

* Cleanup imports

- Alphabetises imports
- Removes unused imports

* Add Platform requirements

* Move filling asset into block where asset exists

* Remove duplicate array keys
2019-03-13 20:12:03 -07:00
snipe b5211b2dd5 Fixed #6291 - send-welcome argument in cli importer 2018-10-04 04:44:12 -07:00
snipe c8ad45b11e Port import manager code to develop 2018-10-03 00:52:29 -07:00
snipe 47ed328f0e Change fire() to handle() for Laravel 5.5 2018-07-25 01:28:44 -07:00
Daniel Meltzer bbffde47f7 Remove conditional related to no longer existant testrun option. (#4079) 2017-09-29 17:31:39 -07:00
Daniel Meltzer f432f98e12 Importer tests + Fixes (#3733)
* Fix Bug in User::generateFormattedNameFromFullName

In a name "John Doe", this method would split it into "John" and " Doe",
Leaving a space in the last name when importing to the database.  Strip
this space.

* Cleanup/fix some item mapping.

Also make some changes to the importer schema to allow for unit testing.
Generate a default item mapping, and then merge that with any custom
mappings.

* Beginning work on importer unit tests.

* Strip out testrun branches from importer.  It added a lot of complexity and was not terribly useful with web importer as it stood, might reconsider down the road however.

* Normalize the mapped keys when using custom field mappings.

* Add test for custom asset import mapping.

* Make all unit tests inherit from a new custom base.

This baseclass currently calls Artisan::migrate() and seeds a Settings
instance.  This fixes unit tests after the autoincrement bits.

* Store requestable as a boolean.  Fixes some import oddities

* Work on tests for accessory importer.

* Test for custom mapping of accessory import, also adjust the internal field for purchase date.

* Update default locale fallback for currency detection

* Fix Reassignable in consumable as well.

* More importer tests and fixes.
2017-07-11 20:37:02 -07:00
Daniel Meltzer 61c6160b98 Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.

* Pull the modal out into a dedicated partial, move importer to views/importer.

* Add document of CSV->importer mappings.  Reorganize some code.

Progress.

* Add header_row and first_row to imports table, and process upon uploading a file

* Use an expandable table row instead of a modal for import processing.  This should allow for field mapping interaction easier.

* Fix import processing after moving method.

* Frontend importer mapping improvements.

Invert display so we show found columns and allow users to select an
importer field to map to.  Also implement sample data based on first row
of csv.

* Update select2.  Maintain selected items properly.

* Backend support for importing.  Only works on the web importer currently.  Definitely needs testing and polish.

* We no longer use vue-modal plugin.

* Add a column to track field mappings to the imports table.

* Cleanup/rename methods+refactor

* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.

* Update debugbar to fix a bug in the debugbar code.

* Fix asset tag detection.

Also rename findMatch to be a bit clearer as to what it does.
  Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.

This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.

* Dynamically limit options for select2 based on import type selected, and group them by item type.

* Add user importer.

Still need to implement emailing of passwords to new users, and probably
test a bit more.

This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.

* Query the db to find user matches rather than search the array.  Performance is much much better.

* Speed/memory improvements in importers.

Move to querying the db rather than maintaining an array for all
importers.  Also only store the id of items when we import, rather than
the full model.  It saves a decent amount of memory.

* Remove grouping of items in select2

With the values being set dynamically, the grouping is redundant.  It
also caused a regression with automatically guessing/matching field
names.  This is starting to get close.

* Remove debug line on every create.

* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb

* Fix asset import regression matching email address.

* Rearrange travis order in attempt to fix null settings.

* Use auth::id instead of fetching it off the user.  Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
snipe af211c1203 Remove unused method 2017-01-25 03:10:44 -08:00
Daniel Meltzer 44683c784f Importer: Add License Importer and refactor (#3143)
* Major code simplification of the importers.

Move towards using Model::fill and Model::update rather than reinventing
the wheel.  This makes the updating/creating logic a lot clearer, and
allows for the deletion of a lot of code.  Also allows for supporting of
more fields in the future really easily.

* Cleanup constructors and use setters instead.

* Set the LC_MONETARY locale, and use it to strip currency symbols in Helper::parseFloat()

* Move licenseseat creation/deletion logic into an event handler on the model rather than the controller.

* Move the logging of parsed values to array_smart_fetch rather than writing it out everywhere

* Move to storing dates as carbon rather than strings.  Allows for the parsing of more arbitrary strings from the importer

* Add a license importer with support for checking out to users or assets.

* Make a directory for sample/mock import csvs and populate it

* Adjust how we store/retrieve dates to fix some issues the tests found.
2017-01-10 16:19:18 -08:00
Daniel Meltzer 6ce20c32b1 Component Importer and various Importer Fixes (#3132)
* Importer fix: we were trimming the wrong part of the classname when creating a category.  This led to categories not being recognized.

* Add a component importer.  Uses same fields as consumable importer.  Only trick: If an asset_tag is present, we checkout a component to that asset on import

Enable component importer.  Also calculate the importer classname in a cleaner fashion.

* Fix comparisons.  find can return an index of 0, which is falsy.
2017-01-05 15:45:12 -08:00
snipe 51ceaedfaf Small phpcbf cleanup 2016-12-29 14:02:18 -08:00
Daniel Meltzer 57374955a8 Importer rework (#3100)
* Step 1 of refactoring importer to use separate classes.

* Port web importer.  Fix an issue with validation where index 0 would be treated as false and cause weird results.

* Farewall, AssetImport.  You've served us well.
2016-12-26 15:16:42 -08:00
snipe d6b7036518 NOT FINISHED: Added case for component import
This is not finished. The actual component creation isn’t written yet
2016-12-22 16:06:55 -08:00
Daniel Meltzer d722ed3823 Partialize forms (#2884)
* Consolidate edit form elements into reusable partials.

This is a large code change that doesn't do much immediately.  It
refactors all of the various edit.blade.php files to reference
standardized partials, so that they all reference the same base html
layout. This has the side effect of moving everything to the new fancy
"required" indicators, and making things look consistent.

In addition, I've gone ahead and renamed a few database fields.  We had
Assetmodel::modelno and Consumable::model_no, I've renamed both to
model_number.  We had items using ::note and ::notes, I've standardized
on ::notes.  Component used total_qty where consumables and accessories
used qty, so I've moved everything to qty (And fixed a few bugs in the
helper file in the process.

TODO includes looking at how/where to place the modal javascripts to
allow for on the fly creation from all places, rather than just the
asset page.

Rename assetmodel::modelno to model_number for clarity and consistency

Rename consumable::model_no to model_number for clarity and consistency

Rename assetmodel::note to notes for clarity and consistency

Port asset and assetmodel to new partials layout.  Adapt all code to the renamed model_number and notes database changes.  Fix some stying.

* Share a settings variable with all views.

* Allow editing the per_page setting.  We showed the value, but we never showed it on the edit page..

* use snipeSettings in all views instead of the long ugly path.

* War on partials. Centralize all bootstrap table javascript

* Use model_number instead of modelno in importer

* Codacy fix.

* More unification/deduplication.  Create an edit form template layout that we use as the base for all edit forms.  This gives the same interface for editing everything and makes the edit.blade.* files much easier to read.

* Use a ViewComposer instead of sharing the variable directly.  Fixes artisan optimize trying to hit the db--which ruins new installs

* Fix DB seeder.

* Base sql dump and csv's to import data from for tests.

* Start some functional tests for creating items.

* Add functional tests for all create methods.  Still need to do tests for edits, deletes, and lots of other things

* Improvements to functional tests.

Use the built in DB seeding mechanism instead of doing it ourselves.
Break the tests into multiple units, rather than testing everything in
each function.

* Some improvements to acceptance tests.

Make sure we're only looking at the "trs" within the bootstrap table.
Creation of assets is now tested at the functional level (and is faster)
so ignore it here.

I'm testing acceptance tests with the
IMPORT_{ASSETS,ACCESSORIES,CONSUMABLES}.csv in the tests/_data folder
imported.

* A few things to make acceptance tests work.  Add a name to the companies table, and make the locations table have the correct name

* Use a .env.tests file for testing functional and unit to allow a separate database.

* Add functional tests for compoents, groups, and licenses.

* Now that the config is in the functional.yml, this just confuses things.

* Start some functional tests for creating items.

* Add functional tests for all create methods.  Still need to do tests for edits, deletes, and lots of other things

* Improvements to functional tests.

Use the built in DB seeding mechanism instead of doing it ourselves.
Break the tests into multiple units, rather than testing everything in
each function.

* Some improvements to acceptance tests.

Make sure we're only looking at the "trs" within the bootstrap table.
Creation of assets is now tested at the functional level (and is faster)
so ignore it here.

I'm testing acceptance tests with the
IMPORT_{ASSETS,ACCESSORIES,CONSUMABLES}.csv in the tests/_data folder
imported.

* update db dump

* Update tests to new reality

* env for the test setup

* only load the database at beginning of tests, not between each Functional test.

* Fix a miss from renaming note to notes.

* Set Termination date when creating an asset.  It was only set on edit.

* Rename serial_number to serial in components for consistency.

* Update validation rules to match limits in database.  Currently we just accepted the values and they were truncated when adding to DB.

* Much more detailed functional testing of creating items.  This checks to make sure all values on form have been successfully persisted to database.
2016-11-16 16:56:57 -08:00
Daniel Meltzer a418dece80 Better checking for empty values when updating. (#2811)
* Better checking for empty values when updating.  There's a lot of conditionals in here that we may want to look at cleaning up over time

* Fix typo.  No manfacturers here.

* Fix model update/import.  Also hardcode the status id of unset assets to the first existing one instead of an id that may not exist... Still not ideal, but better.

* Let requests to .env through the middleware.  We check to see if this is readable during setup as a warning, and as it stands it triggers an infinite loop trying to hit the file.
2016-10-25 19:51:13 -07:00
snipe 59ce46dce7 Merge branch 'hotfixes/use_modelno_if_no_name_in_import' into develop 2016-10-25 03:53:45 -07:00
snipe 2dbd1089f5 Use model number if model name is blank 2016-10-25 03:53:19 -07:00
Daniel Meltzer e85241af6a Log checkouts (#2772)
* Ensure the log has a target before trying to fetch the associated company

* Log creation of items, both in importer and manually
2016-10-12 18:45:32 -07:00
snipe 2cfb015b1a Removed extra debugging info 2016-08-29 23:13:30 -07:00
snipe e4dcd47d6c Fix UTF-8 issues in imports 2016-08-29 22:57:48 -07:00
Daniel Meltzer 276e0a7114 Importer: Implement item update and interface improvments (#2507)
* Add support for updating assets to the importer.

If an asset with a matching asset tag is found, and the --update flag is
passed to the importer, we edit the matching asset with any
asset-specific values, and persist to the database.  Any missing/blank
values are skipped.

TODO: Add to web interface, add support in consumables/accessories

* Allow deleting of files on the import page.

* Extend web interface to allow updating of imported items.

This adds a modal dialog to the import process.  Currently the dialog
allows the choice of update vs ignore, and choosing the item type to
import (Accessory, Asset, Consumable).

Also use Helper::ParseFloat() for purchase_cost processing.  It exists,
and fixes issues on my end at least.

* Implement editing of consumables and accessories.

* Rename getProcessImportFile to postProcessImportFile to reflect how it's now used

* Fix copy-pasta error.
2016-08-29 15:49:32 -07:00
snipe 4cf2d16c7a Temp fix for number import on cost 2016-08-23 19:01:40 -07:00
snipe 6fe639f490 Skip email if no email domain is given 2016-08-02 12:30:02 -07:00
snipe a5467ba25d Slug the username to handle spaces, etc. 2016-08-02 11:12:28 -07:00
snipe 987b969e88 Move bcrypt out of loop for object importer 2016-07-25 22:31:33 -07:00
snipe 52425f62c5 Set default values for status labels in import 2016-07-19 15:49:04 -07:00
Daniel Meltzer 68d8e6a3a9 Importer again (#2260)
* Quote item names when displaying in error table.

* Remove the danger class as well as the warning class when uploading a file.  Fixes a display error if a file is successful after a file fails upload.
2016-07-18 15:46:03 -07:00
snipe 8d71e6b268 Fixed indenting 2016-07-09 18:20:57 -07:00
Daniel Meltzer 25b916afeb Show the name of the item with errors in the table. 2016-06-22 17:09:04 -04:00
snipe 4ed8ff5576 Formatting fixes for coding standards 2016-06-22 12:27:41 -07:00
Daniel Meltzer 43e64a5f8c Store a list of errors associated with each field instead of overwriting the error if multiple occur. Also Adjust the importer to extract the asset tag earlier in the process so all asset related errors are associated with the asset tag rather than some being tied to the name. 2016-06-22 15:13:01 -04:00
snipe 49bf9f8b5b Switch null to lowercase for code style adherence 2016-06-21 11:06:53 -07:00
Daniel Meltzer d6dd449445 Only show importer progress if on command line. It broke error display from web. 2016-06-20 00:09:17 -04:00
Daniel Meltzer 8c00bad6d0 Add a check to ensure custom fields exist before iterating through them 2016-06-20 00:09:17 -04:00
Daniel Meltzer 4715cc6447 Pass user id to importer. This shows items imported through web interface as created by the appropriate user. Also save warranty_months to item, not just read it from csv. Fixes #2175 2016-06-20 00:09:17 -04:00
snipe b4d6842a7c Minor formatting changes, added DB call for custom fields 2016-06-13 21:45:52 -07:00
snipe f3e42384c5 Hardcoded higher PHP execution limits 2016-06-13 21:41:09 -07:00
snipe 6fa4d1252d Fixes purchase date if no purchase date given 2016-06-10 05:00:49 -07:00
Daniel Meltzer da01591ab8 Case insensitive string comparison for items when comparing with existing items. Fixes some false errors. Also remove stray characters from manufacturer logging. 2016-06-01 19:31:24 -05:00
Daniel Meltzer a729038ccd Fix typo. 2016-05-31 19:00:11 -05:00
Daniel Meltzer dab2a50982 Only display errors in import table if things actually prevent the asset from being created. Also some table formatting improvements. 2016-05-31 18:04:01 -05:00
Daniel Meltzer 004c63cd5d Improve web imported. Show a list of all items with that were not imported correctly after import. Modify ObjectImporter and add a web-import parameter that causes it to spit out json errors exclusively. Long term I want to separate the console command and the logic so we aren't calling the console command directly, but rather a class that does everything. This would allow for easier progress reports and ajaxification. 2016-05-26 21:29:29 -05:00
Daniel Meltzer 891c37b2ed Add warranty months. Also fix a merge issue where things reverted back to comment instead of log. 2016-05-25 21:13:26 -05:00
Daniel Meltzer e353df588f Importer improvements. If user real name doesn't exist don't try to create the user. 2016-05-25 20:47:46 -05:00
Daniel Meltzer fd0d04eba4 Import status labels. 2016-05-25 20:46:01 -05:00
Daniel Meltzer 22c6f32e92 Instead of a field in the csv, set whether assets, consumables, or accessories are generated by an option on the command line. Also Adjustconsumables/accessories to function smoother. 2016-05-25 20:37:19 -05:00
Daniel Meltzer 04428d2d07 Replace console output in ObjectImporter with a logging mechanism. Also track all model/validation errors and output them at the end of the import, rather than inline. 2016-05-25 20:37:19 -05:00
Daniel Meltzer 592ce4c2de Readd sample.csv. Also port away from Sentry in the importer to be laravel5 compliant. 2016-05-18 23:33:18 -05:00