2016-12-26 15:16:42 -08:00
|
|
|
<?php
|
|
|
|
|
|
|
|
namespace App\Http\Requests;
|
|
|
|
|
2017-01-25 21:29:23 -08:00
|
|
|
use App\Models\Import;
|
2016-12-26 15:16:42 -08:00
|
|
|
use Illuminate\Foundation\Http\FormRequest;
|
|
|
|
use Illuminate\Support\Facades\Auth;
|
|
|
|
|
|
|
|
class ItemImportRequest extends FormRequest
|
|
|
|
{
|
|
|
|
/**
|
|
|
|
* Determine if the user is authorized to make this request.
|
|
|
|
*
|
|
|
|
* @return bool
|
|
|
|
*/
|
|
|
|
public function authorize()
|
|
|
|
{
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/**
|
|
|
|
* Get the validation rules that apply to the request.
|
|
|
|
*
|
|
|
|
* @return array
|
|
|
|
*/
|
|
|
|
public function rules()
|
|
|
|
{
|
|
|
|
return [
|
2017-12-28 20:08:45 -08:00
|
|
|
'import-type' => 'required',
|
2016-12-26 15:16:42 -08:00
|
|
|
];
|
|
|
|
}
|
|
|
|
|
2017-01-25 21:29:23 -08:00
|
|
|
public function import(Import $import)
|
2016-12-26 15:16:42 -08:00
|
|
|
{
|
Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.
* Pull the modal out into a dedicated partial, move importer to views/importer.
* Add document of CSV->importer mappings. Reorganize some code.
Progress.
* Add header_row and first_row to imports table, and process upon uploading a file
* Use an expandable table row instead of a modal for import processing. This should allow for field mapping interaction easier.
* Fix import processing after moving method.
* Frontend importer mapping improvements.
Invert display so we show found columns and allow users to select an
importer field to map to. Also implement sample data based on first row
of csv.
* Update select2. Maintain selected items properly.
* Backend support for importing. Only works on the web importer currently. Definitely needs testing and polish.
* We no longer use vue-modal plugin.
* Add a column to track field mappings to the imports table.
* Cleanup/rename methods+refactor
* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.
* Update debugbar to fix a bug in the debugbar code.
* Fix asset tag detection.
Also rename findMatch to be a bit clearer as to what it does.
Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.
This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.
* Dynamically limit options for select2 based on import type selected, and group them by item type.
* Add user importer.
Still need to implement emailing of passwords to new users, and probably
test a bit more.
This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.
* Query the db to find user matches rather than search the array. Performance is much much better.
* Speed/memory improvements in importers.
Move to querying the db rather than maintaining an array for all
importers. Also only store the id of items when we import, rather than
the full model. It saves a decent amount of memory.
* Remove grouping of items in select2
With the values being set dynamically, the grouping is redundant. It
also caused a regression with automatically guessing/matching field
names. This is starting to get close.
* Remove debug line on every create.
* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb
* Fix asset import regression matching email address.
* Rearrange travis order in attempt to fix null settings.
* Use auth::id instead of fetching it off the user. Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
|
|
|
ini_set('max_execution_time', 600); //600 seconds = 10 minutes
|
|
|
|
ini_set('memory_limit', '500M');
|
2017-01-25 21:29:23 -08:00
|
|
|
$filename = config('app.private_uploads') . '/imports/' . $import->file_path;
|
Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.
* Pull the modal out into a dedicated partial, move importer to views/importer.
* Add document of CSV->importer mappings. Reorganize some code.
Progress.
* Add header_row and first_row to imports table, and process upon uploading a file
* Use an expandable table row instead of a modal for import processing. This should allow for field mapping interaction easier.
* Fix import processing after moving method.
* Frontend importer mapping improvements.
Invert display so we show found columns and allow users to select an
importer field to map to. Also implement sample data based on first row
of csv.
* Update select2. Maintain selected items properly.
* Backend support for importing. Only works on the web importer currently. Definitely needs testing and polish.
* We no longer use vue-modal plugin.
* Add a column to track field mappings to the imports table.
* Cleanup/rename methods+refactor
* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.
* Update debugbar to fix a bug in the debugbar code.
* Fix asset tag detection.
Also rename findMatch to be a bit clearer as to what it does.
Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.
This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.
* Dynamically limit options for select2 based on import type selected, and group them by item type.
* Add user importer.
Still need to implement emailing of passwords to new users, and probably
test a bit more.
This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.
* Query the db to find user matches rather than search the array. Performance is much much better.
* Speed/memory improvements in importers.
Move to querying the db rather than maintaining an array for all
importers. Also only store the id of items when we import, rather than
the full model. It saves a decent amount of memory.
* Remove grouping of items in select2
With the values being set dynamically, the grouping is redundant. It
also caused a regression with automatically guessing/matching field
names. This is starting to get close.
* Remove debug line on every create.
* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb
* Fix asset import regression matching email address.
* Rearrange travis order in attempt to fix null settings.
* Use auth::id instead of fetching it off the user. Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
|
|
|
$import->import_type = $this->input('import-type');
|
|
|
|
$class = title_case($import->import_type);
|
2017-01-05 15:45:12 -08:00
|
|
|
$classString = "App\\Importer\\{$class}Importer";
|
2017-01-10 16:19:18 -08:00
|
|
|
$importer = new $classString($filename);
|
Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.
* Pull the modal out into a dedicated partial, move importer to views/importer.
* Add document of CSV->importer mappings. Reorganize some code.
Progress.
* Add header_row and first_row to imports table, and process upon uploading a file
* Use an expandable table row instead of a modal for import processing. This should allow for field mapping interaction easier.
* Fix import processing after moving method.
* Frontend importer mapping improvements.
Invert display so we show found columns and allow users to select an
importer field to map to. Also implement sample data based on first row
of csv.
* Update select2. Maintain selected items properly.
* Backend support for importing. Only works on the web importer currently. Definitely needs testing and polish.
* We no longer use vue-modal plugin.
* Add a column to track field mappings to the imports table.
* Cleanup/rename methods+refactor
* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.
* Update debugbar to fix a bug in the debugbar code.
* Fix asset tag detection.
Also rename findMatch to be a bit clearer as to what it does.
Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.
This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.
* Dynamically limit options for select2 based on import type selected, and group them by item type.
* Add user importer.
Still need to implement emailing of passwords to new users, and probably
test a bit more.
This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.
* Query the db to find user matches rather than search the array. Performance is much much better.
* Speed/memory improvements in importers.
Move to querying the db rather than maintaining an array for all
importers. Also only store the id of items when we import, rather than
the full model. It saves a decent amount of memory.
* Remove grouping of items in select2
With the values being set dynamically, the grouping is redundant. It
also caused a regression with automatically guessing/matching field
names. This is starting to get close.
* Remove debug line on every create.
* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb
* Fix asset import regression matching email address.
* Rearrange travis order in attempt to fix null settings.
* Use auth::id instead of fetching it off the user. Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
|
|
|
$import->field_map = request('column-mappings');
|
|
|
|
$import->save();
|
|
|
|
$fieldMappings=[];
|
|
|
|
if ($import->field_map) {
|
|
|
|
// We submit as csv field: column, but the importer is happier if we flip it here.
|
|
|
|
$fieldMappings = array_change_key_case(array_flip($import->field_map), CASE_LOWER);
|
|
|
|
// dd($fieldMappings);
|
|
|
|
}
|
2017-01-10 16:19:18 -08:00
|
|
|
$importer->setCallbacks([$this, 'log'], [$this, 'progress'], [$this, 'errorCallback'])
|
|
|
|
->setUserId(Auth::id())
|
|
|
|
->setUpdating($this->has('import-update'))
|
2018-10-03 00:52:29 -07:00
|
|
|
->setShouldNotify($this->has('send-welcome'))
|
Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.
* Pull the modal out into a dedicated partial, move importer to views/importer.
* Add document of CSV->importer mappings. Reorganize some code.
Progress.
* Add header_row and first_row to imports table, and process upon uploading a file
* Use an expandable table row instead of a modal for import processing. This should allow for field mapping interaction easier.
* Fix import processing after moving method.
* Frontend importer mapping improvements.
Invert display so we show found columns and allow users to select an
importer field to map to. Also implement sample data based on first row
of csv.
* Update select2. Maintain selected items properly.
* Backend support for importing. Only works on the web importer currently. Definitely needs testing and polish.
* We no longer use vue-modal plugin.
* Add a column to track field mappings to the imports table.
* Cleanup/rename methods+refactor
* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.
* Update debugbar to fix a bug in the debugbar code.
* Fix asset tag detection.
Also rename findMatch to be a bit clearer as to what it does.
Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.
This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.
* Dynamically limit options for select2 based on import type selected, and group them by item type.
* Add user importer.
Still need to implement emailing of passwords to new users, and probably
test a bit more.
This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.
* Query the db to find user matches rather than search the array. Performance is much much better.
* Speed/memory improvements in importers.
Move to querying the db rather than maintaining an array for all
importers. Also only store the id of items when we import, rather than
the full model. It saves a decent amount of memory.
* Remove grouping of items in select2
With the values being set dynamically, the grouping is redundant. It
also caused a regression with automatically guessing/matching field
names. This is starting to get close.
* Remove debug line on every create.
* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb
* Fix asset import regression matching email address.
* Rearrange travis order in attempt to fix null settings.
* Use auth::id instead of fetching it off the user. Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
|
|
|
->setUsernameFormat('firstname.lastname')
|
|
|
|
->setFieldMappings($fieldMappings);
|
|
|
|
// $logFile = storage_path('logs/importer.log');
|
|
|
|
// \Log::useFiles($logFile);
|
2016-12-26 15:16:42 -08:00
|
|
|
$importer->import();
|
|
|
|
return $this->errors;
|
|
|
|
}
|
|
|
|
|
|
|
|
public function log($string)
|
|
|
|
{
|
Importer mapping - v1 (#3677)
* Move importer to an inline-template, allows for translations and easier passing of data from laravel to vue.
* Pull the modal out into a dedicated partial, move importer to views/importer.
* Add document of CSV->importer mappings. Reorganize some code.
Progress.
* Add header_row and first_row to imports table, and process upon uploading a file
* Use an expandable table row instead of a modal for import processing. This should allow for field mapping interaction easier.
* Fix import processing after moving method.
* Frontend importer mapping improvements.
Invert display so we show found columns and allow users to select an
importer field to map to. Also implement sample data based on first row
of csv.
* Update select2. Maintain selected items properly.
* Backend support for importing. Only works on the web importer currently. Definitely needs testing and polish.
* We no longer use vue-modal plugin.
* Add a column to track field mappings to the imports table.
* Cleanup/rename methods+refactor
* Save field mappings and import type when attempting an import, and repopulate these values when returning to the page.
* Update debugbar to fix a bug in the debugbar code.
* Fix asset tag detection.
Also rename findMatch to be a bit clearer as to what it does.
Remove logging to file of imports for http imports because
it eats an incredible amouint of memory.
This commit also moves imports out of the hardware namespace and into
their own webcontroller and route prefix, remove dead code from
AssetController as a result.
* Dynamically limit options for select2 based on import type selected, and group them by item type.
* Add user importer.
Still need to implement emailing of passwords to new users, and probably
test a bit more.
This also bumps the memory limit for web imports up as well, I need to
profile memory usage here before too long.
* Query the db to find user matches rather than search the array. Performance is much much better.
* Speed/memory improvements in importers.
Move to querying the db rather than maintaining an array for all
importers. Also only store the id of items when we import, rather than
the full model. It saves a decent amount of memory.
* Remove grouping of items in select2
With the values being set dynamically, the grouping is redundant. It
also caused a regression with automatically guessing/matching field
names. This is starting to get close.
* Remove debug line on every create.
* Switch migration to be text field instead of json field for compatibility with older mysql/mariadb
* Fix asset import regression matching email address.
* Rearrange travis order in attempt to fix null settings.
* Use auth::id instead of fetching it off the user. Fixes a null object reference during seeding.
2017-06-21 16:37:37 -07:00
|
|
|
// \Log::Info($string);
|
2016-12-26 15:16:42 -08:00
|
|
|
}
|
|
|
|
|
|
|
|
public function progress($count)
|
|
|
|
{
|
|
|
|
// Open for future
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
public function errorCallback($item, $field, $errorString)
|
|
|
|
{
|
|
|
|
$this->errors[$item->name][$field] = $errorString;
|
|
|
|
}
|
|
|
|
|
|
|
|
private $errors;
|
|
|
|
}
|