We use molecular dynamics simulation to study the aggregation of polymer chains in the environment with or without Lennard-Jones (L-J) fluid consisting of L-J molecules. The neighboring monomers of a polymer chain are connected by rigid bonds and have small or zero bending- and torsion-angle dependent potentials. When the fluid is present, a small number of ``linker'' sites along a polymer chain are randomly assigned to be fluid-attractive, in contrast to the remaining fluid-repelling monomers. We find that the polymer chains tend to aggregate in both cases with or without linker sites, but there are some differences in the structure and the dynamics of aggregation of polymer chains, which suggest the relevance of considering the effects of heterogeneity along the backbone. For the pure system without linker sites and L-J fluid in the environment, we also examine the effects on the aggregation process of changing the strengths of angle potentials or the length of polymer chains. We observe the presence of constrained sphere packing under the chain connectivity, which may be considered as a basic growth mode, common to the aggregations of polypeptides or other polymer-based soft matter materials. Our results are useful for constructing further models for studying and understanding key factors which affect the aggregation of peptides or proteins so that one can design a strategy or drugs to avoid the aggregation of peptides or proteins in the human body.